Managing incident response operations based on monitored network activity

ABSTRACT

Embodiments are directed to monitoring network traffic associated with networks to provide metrics. A monitoring engine may determine an anomaly based on the metrics exceeding threshold values. An inference engine may be instantiated to provide an anomaly profile based on portions of the network traffic that are associated with the anomaly. The inference engine may provide an investigation profile based on the anomaly profile such that the investigation profile includes information associated with investigation activities associated with an investigation of the anomaly. The inference engine may monitor the investigation of the anomaly based on other portions of the network traffic such that the other portions of the network traffic are associated with monitoring an occurrence of the investigation activities. The inference engine may modify a performance score associated with the investigation profile based on the occurrence of the investigation activities and a completion status of the investigation.

TECHNICAL FIELD

The present invention relates generally to network monitoring, and moreparticularly, but not exclusively, to monitoring networks in adistributed network monitoring environment.

BACKGROUND

On most computer networks, bits of data arranged in bytes are packagedinto collections of bytes called packets. These packets are generallycommunicated between computing devices over networks in a wired orwireless manner. A suite of communication protocols is typicallyemployed to communicate between at least two endpoints over one or morenetworks. The protocols are typically layered on top of one another toform a protocol stack. One model for a network communication protocolstack is the Open Systems Interconnection (OSI) model, which definesseven layers of different protocols that cooperatively enablecommunication over a network. The OSI model layers are arranged in thefollowing order: Physical (1), Data Link (2), Network (3), Transport(4), Session (5), Presentation (6), and Application (7).

Another model for a network communication protocol stack is the InternetProtocol (IP) model, which is also known as the Transmission ControlProtocol/Internet Protocol (TCP/IP) model. The TCP/IP model is similarto the OSI model except that it defines four layers instead of seven.The TCP/IP model's four layers for network communication protocol arearranged in the following order: Link (1), Internet (2), Transport (3),and Application (4). To reduce the number of layers from four to seven,the TCP/IP model collapses the OSI model's Application, Presentation,and Session layers into its Application layer. Also, the OSI's Physicallayer is either assumed or is collapsed into the TCP/IP model's Linklayer. Although some communication protocols may be listed at differentnumbered or named layers of the TCP/IP model versus the OSI model, bothof these models describe stacks that include basically the sameprotocols. For example, the TCP protocol is listed on the fourth layerof the OSI model and on the third layer of the TCP/IP model. To assessand troubleshoot communicated packets and protocols over a network,different types of network monitors can be employed. One type of networkmonitor, a “packet sniffer” may be employed to generally monitor andrecord packets of data as they are communicated over a network. Somepacket sniffers can display data included in each packet and providestatistics regarding a monitored stream of packets. Also, some types ofnetwork monitors are referred to as “protocol analyzers” in part becausethey can provide additional analysis of monitored and recorded packetsregarding a type of network, communication protocol, or application.

Generally, packet sniffers and protocol analyzers passively monitornetwork traffic without participating in the communication protocols. Insome instances, they receive a copy of each packet on a particularnetwork segment or VLAN from one or more members of the network segment.They may receive these packet copies through a port mirror on a managedEthernet switch, e.g., a Switched Port Analyzer (SPAN) port, a RovingAnalysis Port (RAP), or the like, or combinations thereof. Portmirroring enables analysis and debugging of network communications. Portmirroring can be performed for inbound or outbound traffic (or both) onsingle or multiple interfaces. In other instances, packet copies may beprovided to the network monitors from a specialized network tap or froma software entity running on the client or server. In virtualenvironments, port mirroring may be performed on a virtual switch thatis incorporated within the hypervisor.

In complex networks, network activity, investigating performance oractivity anomalies may difficult given the complexity and size ofcontemporary networks. This may result in incident analysts performingad-hoc actions to resolve or investigate anomalies in the network. Also,in some cases, organizations may provide workflows or playbooks that tohelp analysts leverage past investigations. However, ensuring the theseworkflows or playbooks are followed may be difficult. Also, in somecases, it may be difficult for an organization to track whichinvestigation workflows or playbooks may be effective. Thus, it is withrespect to these considerations and others that the present inventionhas been made.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments of the present innovationsare described with reference to the following drawings. In the drawings,like reference numerals refer to like parts throughout the variousfigures unless otherwise specified. For a better understanding of thedescribed innovations, reference will be made to the following DetailedDescription of Various Embodiments, which is to be read in associationwith the accompanying drawings, wherein:

FIG. 1 illustrates a system environment in which various embodiments maybe implemented;

FIG. 2 illustrates a schematic embodiment of a client computer;

FIG. 3 illustrates a schematic embodiment of a network computer;

FIG. 4 illustrates a logical architecture of a system for managingincident response operations based on network activity in accordancewith one or more of the various embodiments;

FIG. 5 illustrates a logical schematic of a system for managing incidentresponse operations based on network activity in accordance with one ormore of the various embodiments;

FIG. 6 illustrates a logical representation of a network in accordancewith at least one of the various embodiments;

FIG. 7 illustrates a logical representation of a portion of a devicerelation model in accordance with at least one of the variousembodiments;

FIG. 8A illustrates a logical representation of a device relation modelshowing naïve relationships between the entities in accordance with theone or more embodiments;

FIG. 8B illustrates a logical representation of a device relation modelshowing informed relationships between the entities in accordance withthe one or more embodiments;

FIG. 9A illustrates a logical representation of a device relation modelshowing relationships between the entities based on observed networkconnections in accordance with the one or more embodiments;

FIG. 9B illustrates a logical representation of a device relation modelshowing phantom edges that represent relationships between the entitiesin accordance with the one or more embodiments;

FIG. 10 illustrates a logical architecture of a network that includesentities in accordance with the one or more embodiments;

FIG. 11 illustrates a logical representation of a data structure for adevice relation model that includes entities in accordance with the oneor more embodiments;

FIG. 12 represents a logical representation of a system for transformingmonitored network traffic into anomaly profile objects (e.g., anomalyprofiles) or investigation profile objects (e.g., investigationprofiles) in accordance with one or more of the various embodiments;

FIG. 13 illustrates a logical schematic of a system for managingincident response operations based on network activity in accordancewith one or more of the various embodiments;

FIG. 14 illustrates an overview flowchart of a process for managingincident response operations based on network activity in accordancewith one or more of the various embodiments;

FIG. 15 illustrates a flowchart of a process for providing anomalyprofiles based on network activity in accordance with one or more of thevarious embodiments;

FIG. 16 illustrates a flowchart of a process for providing investigationprofiles based on network activity in accordance with one or more of thevarious embodiments;

FIG. 17 illustrates a flowchart of a process for managing incidentresponse operations based on network activity using anomaly profiles andinvestigation profiles in accordance with one or more of the variousembodiments;

FIG. 18 illustrates a flowchart of a process for managing incidentresponse operations based on network activity in accordance with one ormore of the various embodiments;

FIG. 19 illustrates a flowchart of a process for training or optimizingimproved investigation profiles based on historical anomaly profileactivity, historical investigation profile activity, and historicalnetwork activity in accordance with one or more of the variousembodiments; and

FIG. 20 illustrates a flowchart of a process for providing investigationmodels based on anomaly profiles, investigation profiles, and networkactivity in accordance with one or more of the various embodiments.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

Various embodiments now will be described more fully hereinafter withreference to the accompanying drawings, which form a part hereof, andwhich show, by way of illustration, specific exemplary embodiments bywhich the invention may be practiced. The embodiments may, however, beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein; rather, these embodiments areprovided so that this disclosure will be thorough and complete, and willfully convey the scope of the embodiments to those skilled in the art.Among other things, the various embodiments may be methods, systems,media or devices. Accordingly, the various embodiments may take the formof an entirely hardware embodiment, an entirely software embodiment oran embodiment combining software and hardware aspects. The followingdetailed description is, therefore, not to be taken in a limiting sense.

Throughout the specification and claims, the following terms take themeanings explicitly associated herein, unless the context clearlydictates otherwise. The phrase “in one embodiment” as used herein doesnot necessarily refer to the same embodiment, though it may.Furthermore, the phrase “in another embodiment” as used herein does notnecessarily refer to a different embodiment, although it may. Thus, asdescribed below, various embodiments may be readily combined, withoutdeparting from the scope or spirit of the invention.

In addition, as used herein, the term “or” is an inclusive “or”operator, and is equivalent to the term “and/or,” unless the contextclearly dictates otherwise. The term “based on” is not exclusive andallows for being based on additional factors not described, unless thecontext clearly dictates otherwise. In addition, throughout thespecification, the meaning of “a,” “an,” and “the” include pluralreferences. The meaning of “in” includes “in” and “on.”

For example embodiments, the following terms are also used hereinaccording to the corresponding meaning, unless the context clearlydictates otherwise.

As used herein the term, “engine” refers to logic embodied in hardwareor software instructions, which can be written in a programminglanguage, such as C, C++, Objective-C, COBOL, Java™, PHP, Perl,JavaScript, Ruby, VBScript, Microsoft .NET™ languages such as C#, or thelike. An engine may be compiled into executable programs or written ininterpreted programming languages. Software engines may be callable fromother engines or from themselves. Engines described herein refer to oneor more logical modules that can be merged with other engines orapplications, or can be divided into sub-engines. The engines can bestored in non-transitory computer-readable medium or computer storagedevice and be stored on and executed by one or more general purposecomputers, thus creating a special purpose computer configured toprovide the engine.

As used herein, the term “session” refers to a semi-permanentinteractive packet interchange between two or more communicatingendpoints, such as network devices. A session is set up or establishedat a certain point in time, and torn down at a later point in time. Anestablished communication session may involve more than one message ineach direction. A session may have stateful communication where at leastone of the communicating network devices saves information about thesession history to be able to communicate. A session may also providestateless communication, where the communication consists of independentrequests with responses between the endpoints. An established session isthe basic requirement to perform a connection-oriented communication. Asession also is the basic step to transmit in connectionlesscommunication modes.

As used herein, the terms “network connection,” and “connection” referto communication sessions with a semi-permanent connection forinteractive packet interchange between two or more communicatingendpoints, such as network devices. The connection may be establishedbefore application data is transferred, and where a stream of data isdelivered in the same or different order than it was sent. Thealternative to connection-oriented transmission is connectionlesscommunication. For example, the datagram mode of communication used bythe Internet Protocol (IP) and the Universal Datagram Protocol (UDP) maydeliver packets out of order, since different packets may be routedindependently and could be delivered over different paths. Packetsassociated with a TCP protocol connection may also be routedindependently and could be delivered over different paths. However, forTCP connections the network communication system may provide the packetsto application endpoints in the correct order.

Connection-oriented communication may be a packet-mode virtual circuitconnection. For example, a transport layer virtual circuit protocol suchas the TCP protocol can deliver packets of data in order although thelower layer switching is connectionless. A connection-oriented transportlayer protocol such as TCP can also provide connection-orientedcommunications over connectionless communication. For example, if TCP isbased on a connectionless network layer protocol (such as IP), thisTCP/IP protocol can then achieve in-order delivery of a byte stream ofdata, by means of segment sequence numbering on the sender side, packetbuffering and data packet reordering on the receiver side.Alternatively, the virtual circuit connection may be established in adatalink layer or network layer switching mode, where all data packetsbelonging to the same traffic stream are delivered over the same path,and traffic flows are identified by some connection identifier ratherthan by complete routing information, which enables fast hardware basedswitching.

As used herein, the terms “session flow” and “network flow” refer to oneor more network packets or a stream of network packets that arecommunicated in a session that is established between at least twoendpoints, such as two network devices. In one or more of the variousembodiments, flows may be useful if one or more of the endpoints of asession may be behind a network traffic management device, such as afirewall, switch, router, load balancer, or the like. In one or more ofthe various embodiments, such flows may be used to ensure that thepackets sent between the endpoints of a flow may be routedappropriately.

Typically, establishing a TCP based connection between endpoints beginswith the execution of an initialization protocol and creates a singlebi-directional flow between two endpoints, e.g., one direction of flowgoing from endpoint A to endpoint B, the other direction of the flowgoing from endpoint B to endpoint A, where each endpoint is at leastidentified by an IP address and a TCP port.

Also, some protocols or network applications may establish a separateflow for control information that enables management of at least one ormore flows between two or more endpoints. Further, in some embodiments,network flows may be half-flows that may be unidirectional.

As used herein, the term “tuple” refers to a set of values that identifya source and destination of a network packet, which may, under somecircumstances, be a part of a network connection. In one embodiment, atuple may include a source Internet Protocol (IP) address, a destinationIP address, a source port number, a destination port number, virtual LANsegment identifier (VLAN ID), tunnel identifier, routing interfaceidentifier, physical interface identifier, or a protocol identifier.Tuples may be used to identify network flows (e.g., connection flows).

As used herein the term “related flows,” or “related network flows” asused herein are network flows that while separate they are operatingcooperatively. For example, some protocols, such as, FTP, SIP, RTP,VOIP, custom protocols, or the like, may provide control communicationover one network flow and data communication over other network flows.Further, configuration rules may define one or more criteria that areused to recognize that two or more network flows should be consideredrelated flows. For example, configuration rules may define that flowscontaining a particular field value should be grouped with other flowshaving the same field value, such as, a cookie value, or the like.

As used herein, the terms “network monitor”, “network monitoringcomputer”, or “NMC” refer to an application (software, hardware, or somecombination) that is arranged to monitor and record flows of packets ina session that are communicated between at least two endpoints over atleast one network. The NMC can provide information for assessingdifferent aspects of these monitored flows. In one or more embodiment,the NMC may passively monitor network packet traffic withoutparticipating in the communication protocols. This monitoring may beperformed for a variety of reasons, including troubleshooting andproactive remediation, end-user experience monitoring, SLA monitoring,capacity planning, application lifecycle management, infrastructurechange management, infrastructure optimization, business intelligence,security, and regulatory compliance. The NMC can receive networkcommunication for monitoring through a variety of means includingnetwork taps, wireless receivers, port mirrors or directed tunnels fromnetwork switches, clients or servers including the endpoints themselves,or other infrastructure devices. In at least some of the variousembodiments, the NMC may receive a copy of each packet on a particularnetwork segment or virtual local area network (VLAN). Also, for at leastsome of the various embodiments, they may receive these packet copiesthrough a port mirror on a managed Ethernet switch, e.g., a SwitchedPort Analyzer (SPAN) port, a Roving Analysis Port (RAP), or the like, orcombination thereof. Port mirroring enables analysis and debugging ofnetwork communications. Port mirroring can be performed for inbound oroutbound traffic (or both) on single or multiple interfaces.

The NMC may track network connections from and to end points such as aclient or a server. The NMC may also extract information from thepackets including protocol information at various layers of thecommunication protocol stack. The NMC may reassemble or reconstruct thestream of data exchanged between the endpoints. The NMC may performdecryption of the payload at various layers of the protocol stack. TheNMC may passively monitor the network traffic or it may participate inthe protocols as a proxy. The NMC may attempt to classify the networktraffic according to communication protocols that are used.

The NMC may also perform one or more actions for classifying protocolsthat may be a necessary precondition for application classification.While some protocols run on well-known ports, others do not. Thus, evenif there is traffic on a well-known port, it is not necessarily theprotocol generally understood to be assigned to that port. As a result,the NMC may perform protocol classification using one or moretechniques, such as, signature matching, statistical analysis, trafficanalysis, and other heuristics. In some cases, the NMC may use adaptiveprotocol classification techniques where information used to classifythe protocols may be accumulated or applied over time to furtherclassify the observed protocols. In some embodiments, NMCs may bearranged to employ stateful analysis. Accordingly, for each supportedprotocols, an NMC may use network packet payload data to drive a statemachine that mimics the protocol state changes in the client/serverflows being monitored. The NMC may categorize the traffic wherecategories might include file transfers, streaming audio, streamingvideo, database access, interactive, gaming, and the like. The NMC mayattempt to determine whether the traffic corresponds to knowncommunications protocols, such as HTTP, FTP, SMTP, RTP, TDS, TCP, IP,and the like.

In one or more of the various embodiments, NMCs or NMC functionality maybe implemented using hardware or software based proxy devices that maybe arranged to intercept network traffic in the monitored networks.

As used herein, the terms “layer” and “model layer” refer to a layer ofone or more communication protocols in a stack of communication protocollayers that are defined by a model, such as the OSI model and the TCP/IP(IP) model. The OSI model defines seven layers and the TCP/IP modeldefines four layers of communication protocols.

For example, at the OSI model's lowest or first layer (Physical),streams of electrical/light/radio impulses (bits) are communicatedbetween computing devices over some type of media, such as cables,network interface cards, radio wave transmitters, and the like. At thenext or second layer (Data Link), bits are encoded into packets andpackets are also decoded into bits. The Data Link layer also has twosub-layers, the Media Access Control (MAC) sub-layer and the LogicalLink Control (LLC) sub-layer. The MAC sub-layer controls how a computingdevice gains access to the data and permission to transmit it. The LLCsub-layer controls frame synchronization, flow control and errorchecking. At the third layer (Network), logical paths are created, knownas virtual circuits, to communicated data from node to node. Routing,forwarding, addressing, internetworking, error handling, congestioncontrol, and packet sequencing are functions of the Network layer. Atthe fourth layer (Transport), transparent transfer of data between endcomputing devices, or hosts, is provided. The Transport layer isresponsible for end to end recovery and flow control to ensure completedata transfer over the network.

At the fifth layer (Session) of the OSI model, connections betweenapplications are established, managed, and terminated. The Session layersets up, coordinates, and terminates conversations, exchanges, anddialogues between applications at each end of a connection. At the sixthlayer (Presentation), independence from differences in datarepresentation, e.g., encryption, is provided by translating fromapplication to network format and vice versa. Generally, thePresentation layer transforms data into the form that the protocols atthe Application layer (7) can accept. For example, the Presentationlayer generally handles the formatting and encrypting/decrypting of datathat is communicated across a network.

At the top or seventh layer (Application) of the OSI model, applicationand end user processes are supported. For example, communicationpartners may be identified, quality of service can be identified, userauthentication and privacy may be considered, and constraints on datasyntax can be identified. Generally, the Application layer providesservices for file transfer, messaging, and displaying data. Protocols atthe Application layer include FTP, HTTP, and Telnet.

To reduce the number of layers from seven to four, the TCP/IP modelcollapses the OSI model's Application, Presentation, and Session layersinto its Application layer. Also, the OSI's Physical layer is eitherassumed or may be collapsed into the TCP/IP model's Link layer. Althoughsome communication protocols may be listed at different numbered ornamed layers of the TCP/IP model versus the OSI model, both of thesemodels describe stacks that include basically the same protocols.

As used herein the term “metric” refers to a value that represents oneor more performance characteristics of a monitored network. Metrics mayinclude aggregated measurements, rate of changes, proportions, or thelike, Metrics may be associated with particular network application,network protocols, entities, or the like. Metrics may include anindication of the presence of one or more patterns, such as, networkpacket headers, protocol preambles, or the like. Common metrics mayinclude: requests and responses for Hypertext Transfer Protocol) HTTP;database protocols; Transport Security Layer/Secure Sockets Layer(TLS/SSL); storage protocols, such as, Common Internet File System(CIFS) or Network File System (NFS), Domain Name Service (DNS),Lightweight Directory Access Protocol (LDAP); NoSQL storage protocolssuch as MongoDB or Memcache, File Transfer Protocol (FTP), Simple MailTransfer Protocol (SMTP); and Voice-over-IP (VoIP) protocols such asSession Initiation Protocol (SIP) and Real-time Transport Protocol(RTP); or the like. Within each of metric, there may be measurementsbroken down by Response Status Codes (that may apply across manyprotocols, including HTTP, SMTP, SIP); HTTP Requests by Method (GET,POST, HEAD, or the like.); requests by SQL method and table, SSLcertificate expiration time by host, various protocols by username(e.g., LDAP, FTP, SMTP, VoIP, or the like), and by client or server IP(almost every protocol). Also, in some embodiments, users may definecustom metrics based on the collection of user-defined measurements fora given protocol or entity.

As used herein the “metric visualization,” or “visualization” refers toa graphical representation of one or more metrics. A metric may beassociated with one or more different types of visualizations. In someembodiments, metric visualizations may be line graphs, pie charts, bargraphs, scatter plots, heat maps, Sankey diagrams, histograms, timeseries graphs, candlestick charts, geolocation charts, or the like, orcombination thereof, displayed in a graphical user interface.

As used herein, the term “entity” refers to an actor or element in amonitored network. Entities may include applications, services,programs, processes, network devices, network computers, clientcomputers, or the like, operating in the monitored network. For example,individual entities may include, web clients, web servers, databaseclients, database servers, mobile app clients, payment processors,groupware clients, groupware services, or the like. In some cases,multiple entities may co-exist on or in the same network computer,process, application, compute container, or cloud compute instance.

As used herein, the term “device relation model” refers to a datastructure that is used to represent relationships between and amongdifferent entities in a monitored network. Device relation models may begraph models comprised of nodes and edges stored in the memory of anetwork computer. In some embodiments, the network computer mayautomatically update the configuration and composition of the devicerelation model stored in the memory of the network computer to reflectthe relationships between two or more entities in the monitored network.Nodes of the graph model may represent entities in the network and theedges of the graph model represent the relationship between entities inthe network. Device relation models may improve the performance ofcomputers at least by enabling a compact representation of entities andrelationships in large networks to reduce memory requirements.

As used herein, the “device profile” refers to a data structure thatrepresents the characteristics of network devices or entities that arediscovered in networks monitored by NMCs. Values or fields in deviceprofiles may be based on metrics, network traffic characteristics,network footprints, or the like, that have been collected based onpassive network monitoring of network traffic in one or more monitorednetworks. Device profiles may be provided for various network devices,such as, client computers, server computers, application servercomputers, networked storage devices, routers, switches, firewalls,virtual machines, cloud instances, or the like.

As used herein, the “application profile” refers to a data structurethat represents the characteristics of applications or services that arediscovered in networks monitored by NMCs. Values or fields inapplication profiles may be based on metrics, network trafficcharacteristics, network footprints, or the like, that have beencollected based on passive network monitoring of network traffic in oneor more monitored networks. Application profiles may be provided forvarious applications, such as, client computers, server computers,application server computers, networked storage devices, routers,switches, firewalls, virtual machines, cloud instances, or the like. Forexample, application profiles may be provided for web clients, webservers, database clients, database servers, credentialing services,mobile application clients, payment processors, groupware clients,groupware services, micro-services, container based services, documentmanagement clients, document management services, billing/invoicingsystems, building management services, healthcare management services,VOIP clients, VOIP servers, or the like.

As used herein, the term “entity profile” refers to a data structurethat represents the characteristics of a network entity that may be acombination of device profiles and application profiles. Entity profilesmay also include additional values or fields based on metrics, networktraffic characteristics, network footprint, or the like, that have beencollected based on passive network monitoring of network traffic in oneor more monitored networks. For example, an entity profile may beprovided for application servers where the entity profile is made fromsome or all of the device profile of the computer running or hosting theapplications and some or all of the application profiles associated withthe applications or services that are running or hosting one thecomputer. In some cases, multiple services or applications running ondevices may be included in the same entity profile. In other cases,entity profiles may be arranged in hierarchal data structure similar toan object oriented computer languages class hierarchy.

As used herein, the term “anomaly profile” refers to a data structurethat that represents the characteristics of particular classes, types,or categorizations of anomalies that may be detected in a monitorednetwork. Anomaly profiles may include various features including valuesor fields based on metrics, network traffic characteristics, activitycontent/traffic, or the like, that have been collected based on passivenetwork monitoring of network traffic in one or more monitored networksbefore, during, or after the occurrence of one or more anomalies.

As used herein, the term “investigation profile” refers to a datastructure that that represents the characteristics of particularclasses, types, or categorizations of activity or actions that wereperformed to investigate one or more anomalies. This may includeactivities directed to devices or entities as well as activity generatedby devices or entities. Investigation profiles may include additionalvalues or fields based on measurements, metrics, network trafficcharacteristics, activity content/traffic, or the like, that have beencollected based on passive network monitoring of network traffic in oneor more monitored networks before, during, or after an investigation ofone or more detected anomalies. In some cases, investigation profilesmay be associated with one or more anomaly profiles, applicationprofiles or entity profiles. In some embodiments, investigation profilesmay include an ordered set of actions or action descriptions that may bepresented to guide investigator that may be investigating one or moreanomalies. Investigation profiles may include actionable information orinstructions that may trigger the display or one or more reports orvisualizations that are related to the investigation of a particularanomaly.

As used herein, the term “observation port” refers to network taps,wireless receivers, port mirrors or directed tunnels from networkswitches, clients or servers, virtual machines, cloud computinginstances, other network infrastructure devices or processes, or thelike, or combination thereof. Observation ports may provide a copy ofeach network packet included in wire traffic on a particular networksegment or virtual local area network (VLAN). Also, for at least some ofthe various embodiments, observation ports may provide NMCs networkpacket copies through a port mirror on a managed Ethernet switch, e.g.,a Switched Port Analyzer (SPAN) port, or a Roving Analysis Port (RAP).

Metrics may include an indication of the presence of one or morepatterns, such as, network packet headers, protocol preambles, or thelike. Common metrics may include: requests and responses for HypertextTransfer Protocol) HTTP; database protocols; Transport SecurityLayer/Secure Sockets Layer (TLS/SSL); storage protocols, such as, CommonInternet File System (CIFS) or Network File System (NFS), Domain NameService (DNS), Lightweight Directory Access Protocol (LDAP); NoSQLstorage protocols such as MongoDB or Memcache, File Transfer Protocol(FTP), Simple Mail Transfer Protocol (SMTP); and Voice-over-IP (VoIP)protocols such as Session Initiation Protocol (SIP) and Real-timeTransport Protocol (RTP); or the like.

Within each of metric, there may be measurements broken down by ResponseStatus Codes (that may apply across many protocols, including HTTP,SMTP, SIP); HTTP Requests by Method (GET, POST, HEAD, or the like.);requests by SQL method and table, SSL certificate expiration time byhost, various protocols by username (e.g., LDAP, FTP, SMTP, VoIP, or thelike), and by client or server IP (almost every protocol). Also, in someembodiments, users may define “custom” metrics based on the collectionof user-defined measurements for a given protocol.

As used herein the “metric visualization” refers to a graphicalrepresentation of a metric. A metric may be associated with one or moredifferent types of visualizations. In some embodiments, metricsvisualizations may be line graphs, pie charts, bar graphs, scatterplots, heat maps, Sankey diagrams, histograms, time series graphs,candlestick charts, geolocation charts, or the like, or combinationthereof, display in a graphical user interface.

The following briefly describes embodiments of the invention in order toprovide a basic understanding of some aspects of the invention. Thisbrief description is not intended as an extensive overview. It is notintended to identify key or critical elements, or to delineate orotherwise narrow the scope. Its purpose is merely to present someconcepts in a simplified form as a prelude to the more detaileddescription that is presented later.

Briefly stated, various embodiments are directed to monitoring networktraffic using one or more network computers. In one or more of thevarious embodiments, a monitoring engine may be instantiated to performvarious actions. In one or more of the various embodiments, themonitoring engine may be arranged to monitor network traffic associatedwith a plurality of entities in one or more networks to provide one ormore metrics. In one or more of the various embodiments, the monitoringengine may be arranged to determine an anomaly based on the one or moremetrics exceeding one or more threshold values.

In one or more of the various embodiments, an inference engine may beinstantiated to perform actions, such as, providing an anomaly profilefrom a plurality of anomaly profiles based on one or more portions ofthe network traffic that are associated with the anomaly. In one or moreof the various embodiments, providing the anomaly profile may include:providing one or more features associated with the anomaly based on theone or more portions of the network traffic that are associated with theanomaly; comparing the one or more features to one or more otherfeatures that are associated with the plurality of anomaly profiles; andgenerating the anomaly profile based on a negative result of thecomparison, wherein the anomaly profile is generated based on the one ormore features.

In one or more of the various embodiments, the inference engine may bearranged to provide an investigation profile from a plurality ofinvestigation profiles based on the anomaly profile such that theinvestigation profile includes information associated with one or moreinvestigation activities associated with an investigation of theanomaly. In one or more of the various embodiments, providing theinvestigation profile may include: providing one or more investigationmodels that are trained to classify anomaly profiles; employing the oneor more investigation models to classifying the anomaly profile; andproviding the investigation profile based on a classification of theanomaly profile.

In one or more of the various embodiments, the inference engine may bearranged to monitor the investigation of the anomaly based on one ormore other portions of the network traffic such that the one or moreother portions of the network traffic are associated with monitoring anoccurrence of the one or more investigation activities.

In one or more of the various embodiments, the inference engine may bearranged to modify a performance score that may be associated with theinvestigation profile based on the occurrence of the one or moreinvestigation activities and a completion status of the investigationsuch that the performance score is decreased when one or more otherinvestigation activities are included in the investigation or when oneor more of the one or more investigation activities are omitted from theinvestigation of the anomaly.

In one or more of the various embodiments, the inference engine may bearranged to: provide network activity information that may be associatedwith one or more previous occurrences of one or more anomalies; provideinvestigation activity information that may be associated with previousinvestigations associated with one or more investigation profiles andthe one or more anomalies; and evaluate the one or more investigationprofiles based on the investigation activity information associated withprevious investigations of the one or more anomalies such that the oneor more investigation profiles may be optimized based on the evaluation.

In one or more of the various embodiments, the inference engine may bearranged to: provide network activity information that may be associatedwith one or more previous occurrences of one or more anomalies; provideinvestigation activity information and completion results that areassociated with previous investigations associated with one or moreinvestigation profiles and the one or more anomalies; train one or moreinvestigation models to provide an investigation profiles based on aninput that includes an input anomaly profile, investigation activityinformation, and completion results; and re-train the one or moreinvestigation models if a confidence score associated with the one ormore investigation models is less than a threshold value.

In one or more of the various embodiments, the inference engine may bearranged to: provide a playbook that defines one or more actions forinvestigating the anomaly; compare the occurrence of the one or moreinvestigation activities with the one or more actions defined in theplaybook to provide a deviation score, such that the deviation score isassociated with a number of the one or more actions that are notperformed during the investigation; and evaluate an efficacy of theplaybook based on the deviation score and the completion resultassociated with the investigation.

In one or more of the various embodiments, the inference engine may bearranged to generate the investigation profile based on the one or moreother portions of the network traffic that are associated with the oneor more investigation activities.

In one or more of the various embodiments, the inference engine may bearranged to provide one or more reports based on the investigation andthe completion result such that the one or more reports are displayed toone or more users.

Illustrated Operating Environment

FIG. 1 shows components of one embodiment of an environment in whichembodiments of the invention may be practiced. Not all of the componentsmay be required to practice the invention, and variations in thearrangement and type of the components may be made without departingfrom the spirit or scope of the invention. As shown, system 100 of FIG.1 includes local area networks (LANs)/wide area networks(WANs)-(network) 110, wireless network 108, client computers 102-105,application server computer 116, network monitoring computer 118, or thelike.

At least one embodiment of client computers 102-105 is described in moredetail below in conjunction with FIG. 2. In one embodiment, at leastsome of client computers 102-105 may operate over one or more wired orwireless networks, such as networks 108, or 110. Generally, clientcomputers 102-105 may include virtually any computer capable ofcommunicating over a network to send and receive information, performvarious online activities, offline actions, or the like. In oneembodiment, one or more of client computers 102-105 may be configured tooperate within a business or other entity to perform a variety ofservices for the business or other entity. For example, client computers102-105 may be configured to operate as a web server, firewall, clientapplication, media player, mobile telephone, game console, desktopcomputer, or the like. However, client computers 102-105 are notconstrained to these services and may also be employed, for example, asfor end-user computing in other embodiments. It should be recognizedthat more or less client computers (as shown in FIG. 1) may be includedwithin a system such as described herein, and embodiments are thereforenot constrained by the number or type of client computers employed.

Computers that may operate as client computer 102 may include computersthat typically connect using a wired or wireless communications mediumsuch as personal computers, multiprocessor systems, microprocessor-basedor programmable electronic devices, network PCs, or the like. In someembodiments, client computers 102-105 may include virtually any portablecomputer capable of connecting to another computer and receivinginformation such as, laptop computer 103, mobile computer 104, tabletcomputers 105, or the like. However, portable computers are not solimited and may also include other portable computers such as cellulartelephones, display pagers, radio frequency (RF) devices, infrared (IR)devices, Personal Digital Assistants (PDAs), handheld computers,wearable computers, integrated devices combining one or more of thepreceding computers, or the like. As such, client computers 102-105typically range widely in terms of capabilities and features. Moreover,client computers 102-105 may access various computing applications,including a browser, or other web-based application.

A web-enabled client computer may include a browser application that isconfigured to send requests and receive responses over the web. Thebrowser application may be configured to receive and display graphics,text, multimedia, and the like, employing virtually any web-basedlanguage. In one embodiment, the browser application is enabled toemploy JavaScript, HyperText Markup Language (HTML), eXtensible MarkupLanguage (XML), JavaScript Object Notation (JSON), Cascading StyleSheets (CS S), or the like, or combination thereof, to display and senda message. In one embodiment, a user of the client computer may employthe browser application to perform various activities over a network(online). However, another application may also be used to performvarious online activities.

Client computers 102-105 also may include at least one other clientapplication that is configured to receive or send content betweenanother computer. The client application may include a capability tosend or receive content, or the like. The client application may furtherprovide information that identifies itself, including a type,capability, name, and the like. In one embodiment, client computers102-105 may uniquely identify themselves through any of a variety ofmechanisms, including an Internet Protocol (IP) address, a phone number,Mobile Identification Number (MIN), an electronic serial number (ESN), aclient certificate, or other device identifier. Such information may beprovided in one or more network packets, or the like, sent between otherclient computers, application server computer 116, network monitoringcomputer 118, or other computers.

Client computers 102-105 may further be configured to include a clientapplication that enables an end-user to log into an end-user accountthat may be managed by another computer, such as application servercomputer 116, network monitoring computer 118, or the like. Such anend-user account, in one non-limiting example, may be configured toenable the end-user to manage one or more online activities, includingin one non-limiting example, project management, software development,system administration, configuration management, search activities,social networking activities, browse various websites, communicate withother users, or the like. Further, client computers may be arranged toenable users to provide configuration information, policy information,or the like, to network monitoring computer 118. Also, client computersmay be arranged to enable users to display reports, interactiveuser-interfaces, results provided by network monitor computer 118, orthe like.

Wireless network 108 is configured to couple client computers 103-105and its components with network 110. Wireless network 108 may includeany of a variety of wireless sub-networks that may further overlaystand-alone ad-hoc networks, and the like, to provide aninfrastructure-oriented connection for client computers 103-105. Suchsub-networks may include mesh networks, Wireless LAN (WLAN) networks,cellular networks, and the like. In one embodiment, the system mayinclude more than one wireless network.

Wireless network 108 may further include an autonomous system ofterminals, gateways, routers, and the like connected by wireless radiolinks, and the like. These connectors may be configured to move freelyand randomly and organize themselves arbitrarily, such that the topologyof wireless network 108 may change rapidly.

Wireless network 108 may further employ a plurality of accesstechnologies including 2nd (2G), 3rd (3G), 4th (4G) 5th (5G) generationradio access for cellular systems, WLAN, Wireless Router (WR) mesh, andthe like. Access technologies such as 2G, 3G, 4G, 5G, and future accessnetworks may enable wide area coverage for mobile computers, such asclient computers 103-105 with various degrees of mobility. In onenon-limiting example, wireless network 108 may enable a radio connectionthrough a radio network access such as Global System for Mobilcommunication (GSM), General Packet Radio Services (GPRS), Enhanced DataGSM Environment (EDGE), code division multiple access (CDMA), timedivision multiple access (TDMA), Wideband Code Division Multiple Access(WCDMA), High Speed Downlink Packet Access (HSDPA), Long Term Evolution(LTE), and the like. In essence, wireless network 108 may includevirtually any wireless communication mechanism by which information maytravel between client computers 103-105 and another computer, network, acloud-based network, a cloud instance, or the like.

Network 110 is configured to couple network computers with othercomputers, including, application server computer 116, networkmonitoring computer 118, client computers 102-105 through wirelessnetwork 108, or the like. Network 110 is enabled to employ any form ofcomputer readable media for communicating information from oneelectronic device to another. Also, network 110 can include the Internetin addition to local area networks (LANs), wide area networks (WANs),direct connections, such as through a universal serial bus (USB) port,Ethernet port, other forms of computer-readable media, or anycombination thereof. On an interconnected set of LANs, including thosebased on differing architectures and protocols, a router acts as a linkbetween LANs, enabling messages to be sent from one to another. Inaddition, communication links within LANs typically include twisted wirepair or coaxial cable, while communication links between networks mayutilize analog telephone lines, full or fractional dedicated digitallines including T1, T2, T3, and T4, or other carrier mechanismsincluding, for example, E-carriers, Integrated Services Digital Networks(ISDNs), Digital Subscriber Lines (DSLs), wireless links includingsatellite links, or other communications links known to those skilled inthe art. Moreover, communication links may further employ any of avariety of digital signaling technologies, including without limit, forexample, DS-0, DS-1, DS-2, DS-3, DS-4, OC-3, OC-12, OC-48, or the like.Furthermore, remote computers and other related electronic devices couldbe remotely connected to either LANs or WANs via a modem and temporarytelephone link. In one embodiment, network 110 may be configured totransport information using one or more network protocols, such InternetProtocol (IP).

Additionally, communication media typically embodies computer readableinstructions, data structures, program modules, or other transportmechanism and includes any information non-transitory delivery media ortransitory delivery media. By way of example, communication mediaincludes wired media such as twisted pair, coaxial cable, fiber optics,wave guides, and other wired media and wireless media such as acoustic,RF, infrared, and other wireless media.

One embodiment of application server computer 116 is described in moredetail below in conjunction with FIG. 3. One embodiment of networkmonitoring computer 118 is described in more detail below in conjunctionwith FIG. 3. Although FIG. 1 illustrates application server computer116, and network monitoring computer 118, each as a single computer, theinnovations or embodiments are not so limited. For example, one or morefunctions of application server computer 116, network monitoringcomputer 118, or the like, may be distributed across one or moredistinct network computers. Moreover, in one or more embodiment, networkmonitoring computer 118 may be implemented using a plurality of networkcomputers. Further, in one or more of the various embodiments,application server computer 116, or network monitoring computer 118 maybe implemented using one or more cloud instances in one or more cloudnetworks. Accordingly, these innovations and embodiments are not to beconstrued as being limited to a single environment, and otherconfigurations, and other architectures are also envisaged.

Illustrative Client Computer

FIG. 2 shows one embodiment of client computer 200 that may include manymore or less components than those shown. Client computer 200 mayrepresent, for example, at least one embodiment of mobile computers orclient computers shown in FIG. 1.

Client computer 200 may include processor 202 in communication withmemory 204 via bus 228. Client computer 200 may also include powersupply 230, network interface 232, audio interface 256, display 250,keypad 252, illuminator 254, video interface 242, input/output interface238, haptic interface 264, global positioning systems (GPS) receiver258, open air gesture interface 260, temperature interface 262,camera(s) 240, projector 246, pointing device interface 266,processor-readable stationary storage device 234, and processor-readableremovable storage device 236. Client computer 200 may optionallycommunicate with a base station (not shown), or directly with anothercomputer. And in one embodiment, although not shown, a gyroscope may beemployed within client computer 200 for measuring or maintaining anorientation of client computer 200.

Power supply 230 may provide power to client computer 200. Arechargeable or non-rechargeable battery may be used to provide power.The power may also be provided by an external power source, such as anAC adapter or a powered docking cradle that supplements or recharges thebattery.

Network interface 232 includes circuitry for coupling client computer200 to one or more networks, and is constructed for use with one or morecommunication protocols and technologies including, but not limited to,protocols and technologies that implement any portion of the OSI modelfor mobile communication (GSM), CDMA, time division multiple access(TDMA), UDP, TCP/IP, SMS, MMS, GPRS, WAP, UWB, WiMax, SIP/RTP, GPRS,EDGE, WCDMA, LTE, UMTS, OFDM, CDMA2000, EV-DO, HSDPA, or any of avariety of other wireless communication protocols. Network interface 232is sometimes known as a transceiver, transceiving device, or networkinterface card (MC).

Audio interface 256 may be arranged to produce and receive audio signalssuch as the sound of a human voice. For example, audio interface 256 maybe coupled to a speaker and microphone (not shown) to enabletelecommunication with others or generate an audio acknowledgement forsome action. A microphone in audio interface 256 can also be used forinput to or control of client computer 200, e.g., using voicerecognition, detecting touch based on sound, and the like.

Display 250 may be a liquid crystal display (LCD), gas plasma,electronic ink, light emitting diode (LED), Organic LED (OLED) or anyother type of light reflective or light transmissive display that can beused with a computer. Display 250 may also include a touch interface 244arranged to receive input from an object such as a stylus or a digitfrom a human hand, and may use resistive, capacitive, surface acousticwave (SAW), infrared, radar, or other technologies to sense touch orgestures.

Projector 246 may be a remote handheld projector or an integratedprojector that is capable of projecting an image on a remote wall or anyother reflective object such as a remote screen.

Video interface 242 may be arranged to capture video images, such as astill photo, a video segment, an infrared video, or the like. Forexample, video interface 242 may be coupled to a digital video camera, aweb-camera, or the like. Video interface 242 may comprise a lens, animage sensor, and other electronics. Image sensors may include acomplementary metal-oxide-semiconductor (CMOS) integrated circuit,charge-coupled device (CCD), or any other integrated circuit for sensinglight.

Keypad 252 may comprise any input device arranged to receive input froma user. For example, keypad 252 may include a push button numeric dial,or a keyboard. Keypad 252 may also include command buttons that areassociated with selecting and sending images.

Illuminator 254 may provide a status indication or provide light.Illuminator 254 may remain active for specific periods of time or inresponse to event messages. For example, when illuminator 254 is active,it may backlight the buttons on keypad 252 and stay on while the clientcomputer is powered. Also, illuminator 254 may backlight these buttonsin various patterns when particular actions are performed, such asdialing another client computer. Illuminator 254 may also cause lightsources positioned within a transparent or translucent case of theclient computer to illuminate in response to actions.

Further, client computer 200 may also comprise hardware security module(HSM) 268 for providing additional tamper resistant safeguards forgenerating, storing or using security/cryptographic information such as,keys, digital certificates, passwords, passphrases, two-factorauthentication information, or the like. In some embodiments, hardwaresecurity module may be employed to support one or more standard publickey infrastructures (PKI), and may be employed to generate, manage, orstore keys pairs, or the like. In some embodiments, HSM 268 may be astand-alone computer, in other cases, HSM 268 may be arranged as ahardware card that may be added to a client computer.

Client computer 200 may also comprise input/output interface 238 forcommunicating with external peripheral devices or other computers suchas other client computers and network computers. The peripheral devicesmay include an audio headset, virtual reality headsets, display screenglasses, remote speaker system, remote speaker and microphone system,and the like. Input/output interface 238 can utilize one or moretechnologies, such as Universal Serial Bus (USB), Infrared, WiFi, WiMax,Bluetooth™, and the like.

Input/output interface 238 may also include one or more sensors fordetermining geolocation information (e.g., GPS), monitoring electricalpower conditions (e.g., voltage sensors, current sensors, frequencysensors, and so on), monitoring weather (e.g., thermostats, barometers,anemometers, humidity detectors, precipitation scales, or the like), orthe like. Sensors may be one or more hardware sensors that collect ormeasure data that is external to client computer 200.

Haptic interface 264 may be arranged to provide tactile feedback to auser of the client computer. For example, the haptic interface 264 maybe employed to vibrate client computer 200 in a particular way whenanother user of a computer is calling. Temperature interface 262 may beused to provide a temperature measurement input or a temperaturechanging output to a user of client computer 200. Open air gestureinterface 260 may sense physical gestures of a user of client computer200, for example, by using single or stereo video cameras, radar, agyroscopic sensor inside a computer held or worn by the user, or thelike. Camera 240 may be used to track physical eye movements of a userof client computer 200.

GPS transceiver 258 can determine the physical coordinates of clientcomputer 200 on the surface of the Earth, which typically outputs alocation as latitude and longitude values. GPS transceiver 258 can alsoemploy other geo-positioning mechanisms, including, but not limited to,triangulation, assisted GPS (AGPS), Enhanced Observed Time Difference(E-OTD), Cell Identifier (CI), Service Area Identifier (SAI), EnhancedTiming Advance (ETA), Base Station Subsystem (BSS), or the like, tofurther determine the physical location of client computer 200 on thesurface of the Earth. It is understood that under different conditions,GPS transceiver 258 can determine a physical location for clientcomputer 200. In one or more embodiment, however, client computer 200may, through other components, provide other information that may beemployed to determine a physical location of the client computer,including for example, a Media Access Control (MAC) address, IP address,and the like.

Human interface components can be peripheral devices that are physicallyseparate from client computer 200, allowing for remote input or outputto client computer 200. For example, information routed as describedhere through human interface components such as display 250 or keyboard252 can instead be routed through network interface 232 to appropriatehuman interface components located remotely. Examples of human interfaceperipheral components that may be remote include, but are not limitedto, audio devices, pointing devices, keypads, displays, cameras,projectors, and the like. These peripheral components may communicateover a Pico Network such as Bluetooth™, Zigbee™ and the like. Onenon-limiting example of a client computer with such peripheral humaninterface components is a wearable computer, which might include aremote pico projector along with one or more cameras that remotelycommunicate with a separately located client computer to sense a user'sgestures toward portions of an image projected by the pico projectoronto a reflected surface such as a wall or the user's hand.

A client computer may include web browser application 226 that isconfigured to receive and to send web pages, web-based messages,graphics, text, multimedia, and the like. The client computer's browserapplication may employ virtually any programming language, including awireless application protocol messages (WAP), and the like. In one ormore embodiment, the browser application is enabled to employ HandheldDevice Markup Language (HDML), Wireless Markup Language (WML),WMLScript, JavaScript, Standard Generalized Markup Language (SGML),HyperText Markup Language (HTML), eXtensible Markup Language (XML),HTMLS, and the like.

Memory 204 may include RAM, ROM, or other types of memory. Memory 204illustrates an example of computer-readable storage media (devices) forstorage of information such as computer-readable instructions, datastructures, program modules or other data. Memory 204 may store BIOS 208for controlling low-level operation of client computer 200. The memorymay also store operating system 206 for controlling the operation ofclient computer 200. It will be appreciated that this component mayinclude a general-purpose operating system such as a version of UNIX, orLINUX′, or a specialized client computer communication operating systemsuch as Windows Phone™, or the Symbian® operating system. The operatingsystem may include, or interface with a Java virtual machine module thatenables control of hardware components or operating system operationsvia Java application programs.

Memory 204 may further include one or more data storage 210, which canbe utilized by client computer 200 to store, among other things,applications 220 or other data. For example, data storage 210 may alsobe employed to store information that describes various capabilities ofclient computer 200. The information may then be provided to anotherdevice or computer based on any of a variety of methods, including beingsent as part of a header during a communication, sent upon request, orthe like. Data storage 210 may also be employed to store socialnetworking information including address books, buddy lists, aliases,user profile information, or the like. Data storage 210 may furtherinclude program code, data, algorithms, and the like, for use by aprocessor, such as processor 202 to execute and perform actions. In oneembodiment, at least some of data storage 210 might also be stored onanother component of client computer 200, including, but not limited to,non-transitory processor-readable removable storage device 236,processor-readable stationary storage device 234, or even external tothe client computer.

Applications 220 may include computer executable instructions which,when executed by client computer 200, transmit, receive, or otherwiseprocess instructions and data. Applications 220 may include, forexample, other client applications 224, web browser 226, or the like.Client computers may be arranged to exchange communications, such as,queries, searches, messages, notification messages, event messages,alerts, performance metrics, log data, API calls, or the like,combination thereof, with application servers or network monitoringcomputers.

Other examples of application programs include calendars, searchprograms, email client applications, IM applications, SMS applications,Voice Over Internet Protocol (VOIP) applications, contact managers, taskmanagers, transcoders, database programs, word processing programs,security applications, spreadsheet programs, games, search programs, andso forth.

Additionally, in one or more embodiments (not shown in the figures),client computer 200 may include one or more embedded logic hardwaredevices instead of CPUs, such as, an Application Specific IntegratedCircuit (ASIC), Field Programmable Gate Array (FPGA), Programmable ArrayLogic (PAL), or the like, or combination thereof. The embedded logichardware devices may directly execute embedded logic to perform actions.Also, in one or more embodiments (not shown in the figures), clientcomputer 200 may include one or more hardware microcontrollers insteadof CPUs. In one or more embodiments, the microcontrollers may directlyexecute their own embedded logic to perform actions and access their owninternal memory and their own external Input and Output Interfaces(e.g., hardware pins or wireless transceivers) to perform actions, suchas System On a Chip (SOC), or the like.

Illustrative Network Computer

FIG. 3 shows one embodiment of network computer 300 that may be includedin a system implementing at least one of the various embodiments.Network computer 300 may include many more or less components than thoseshown in FIG. 3. However, the components shown are sufficient todisclose an illustrative embodiment for practicing these innovations.Network computer 300 may represent, for example, one embodiment of oneor more of application server computer 116, or network monitoringcomputer 118 of FIG. 1.

As shown in the figure, network computer 300 includes a processor 302that may be in communication with a memory 304 via a bus 328. In someembodiments, processor 302 may be comprised of one or more hardwareprocessors, or one or more processor cores. In some cases, one or moreof the one or more processors may be specialized processors designed toperform one or more specialized actions, such as, those describedherein. Network computer 300 also includes a power supply 330, networkinterface 332, audio interface 356, display 350, keyboard 352,input/output interface 338, processor-readable stationary storage device334, and processor-readable removable storage device 336. Power supply330 provides power to network computer 300.

Network interface 332 includes circuitry for coupling network computer300 to one or more networks, and is constructed for use with one or morecommunication protocols and technologies including, but not limited to,protocols and technologies that implement any portion of the OpenSystems Interconnection model (OSI model), global system for mobilecommunication (GSM), code division multiple access (CDMA), time divisionmultiple access (TDMA), user datagram protocol (UDP), transmissioncontrol protocol/Internet protocol (TCP/IP), Short Message Service(SMS), Multimedia Messaging Service (MMS), general packet radio service(GPRS), WAP, ultra-wide band (UWB), IEEE 802.16 WorldwideInteroperability for Microwave Access (WiMax), Session InitiationProtocol/Real-time Transport Protocol (SIP/RTP), or any of a variety ofother wired and wireless communication protocols. Network interface 332is sometimes known as a transceiver, transceiving device, or networkinterface card (NIC). Network computer 300 may optionally communicatewith a base station (not shown), or directly with another computer.

Audio interface 356 is arranged to produce and receive audio signalssuch as the sound of a human voice. For example, audio interface 356 maybe coupled to a speaker and microphone (not shown) to enabletelecommunication with others or generate an audio acknowledgement forsome action. A microphone in audio interface 356 can also be used forinput to or control of network computer 300, for example, using voicerecognition.

Display 350 may be a liquid crystal display (LCD), gas plasma,electronic ink, light emitting diode (LED), Organic LED (OLED) or anyother type of light reflective or light transmissive display that can beused with a computer. In some embodiments, display 350 may be a handheldprojector or pico projector capable of projecting an image on a wall orother object.

Network computer 300 may also comprise input/output interface 338 forcommunicating with external devices or computers not shown in FIG. 3.Input/output interface 338 can utilize one or more wired or wirelesscommunication technologies, such as USB™, Firewire™, WiFi, WiMax,Thunderbolt™, Infrared, Bluetooth™, Zigbee™, serial port, parallel port,and the like.

Also, input/output interface 338 may also include one or more sensorsfor determining geolocation information (e.g., GPS), monitoringelectrical power conditions (e.g., voltage sensors, current sensors,frequency sensors, and so on), monitoring weather (e.g., thermostats,barometers, anemometers, humidity detectors, precipitation scales, orthe like), or the like. Sensors may be one or more hardware sensors thatcollect or measure data that is external to network computer 300. Humaninterface components can be physically separate from network computer300, allowing for remote input or output to network computer 300. Forexample, information routed as described here through human interfacecomponents such as display 350 or keyboard 352 can instead be routedthrough the network interface 332 to appropriate human interfacecomponents located elsewhere on the network. Human interface componentsinclude any component that allows the computer to take input from, orsend output to, a human user of a computer. Accordingly, pointingdevices such as mice, styluses, track balls, or the like, maycommunicate through pointing device interface 358 to receive user input.

GPS transceiver 340 can determine the physical coordinates of networkcomputer 300 on the surface of the Earth, which typically outputs alocation as latitude and longitude values. GPS transceiver 340 can alsoemploy other geo-positioning mechanisms, including, but not limited to,triangulation, assisted GPS (AGPS), Enhanced Observed Time Difference(E-OTD), Cell Identifier (CI), Service Area Identifier (SAI), EnhancedTiming Advance (ETA), Base Station Subsystem (BSS), or the like, tofurther determine the physical location of network computer 300 on thesurface of the Earth. It is understood that under different conditions,GPS transceiver 340 can determine a physical location for networkcomputer 300. In one or more embodiment, however, network computer 300may, through other components, provide other information that may beemployed to determine a physical location of the client computer,including for example, a Media Access Control (MAC) address, IP address,and the like.

In at least one of the various embodiments, applications, such as,operating system 306, network monitoring engine 322, inference engine324, analysis engine 326, anomaly engine 327, web services 329, or thelike, may be arranged to employ geo-location information to select oneor more localization features, such as, time zones, languages,currencies, calendar formatting, or the like. Localization features maybe used when interpreting network traffic, monitoring applicationprotocols, user-interfaces, reports, as well as internal processes ordatabases. In at least one of the various embodiments, geo-locationinformation used for selecting localization information may be providedby GPS 340. Also, in some embodiments, geolocation information mayinclude information provided using one or more geolocation protocolsover the networks, such as, wireless network 108 or network 111.

Memory 304 may include Random Access Memory (RAM), Read-Only Memory(ROM), or other types of memory. Memory 304 illustrates an example ofcomputer-readable storage media (devices) for storage of informationsuch as computer-readable instructions, data structures, program modulesor other data. Memory 304 stores a basic input/output system (BIOS) 308for controlling low-level operation of network computer 300. The memoryalso stores an operating system 306 for controlling the operation ofnetwork computer 300. It will be appreciated that this component mayinclude a general-purpose operating system such as a version of UNIX, orLINUX™, or a specialized operating system such as MicrosoftCorporation's Windows® operating system, or the Apple Corporation's IOS®operating system. The operating system may include, or interface with aJava virtual machine module that enables control of hardware componentsor operating system operations via Java application programs. Likewise,other runtime environments may be included.

Memory 304 may further include one or more data storage 310, which canbe utilized by network computer 300 to store, among other things,applications 320 or other data. For example, data storage 310 may alsobe employed to store information that describes various capabilities ofnetwork computer 300. The information may then be provided to anotherdevice or computer based on any of a variety of methods, including beingsent as part of a header during a communication, sent upon request, orthe like. Data storage 310 may also be employed to store socialnetworking information including address books, buddy lists, aliases,user profile information, or the like. Data storage 310 may furtherinclude program code, data, algorithms, and the like, for use by aprocessor, such as processor 302 to execute and perform actions such asthose actions described below. In one embodiment, at least some of datastorage 310 might also be stored on another component of networkcomputer 300, including, but not limited to, non-transitory media insideprocessor-readable removable storage device 336, processor-readablestationary storage device 334, or any other computer-readable storagedevice within network computer 300, or even external to network computer300. Data storage 310 may include, for example, profiles 312, networktopology database 314, protocol information 316, or the like. profiles312 may be a database arranged for storing the various profiles that areassociated with network activity, entities, anomalies, or the like,including entity profiles, device profiles, application profiles,anomaly profiles, investigation profiles, or the like, that may occur inmonitored networks. Network topology database 314 may be a data storethat contains information related to the topology of one or more networkmonitored by a NMC, including one or more device relation models. And,protocol information 316 may store various rules or configurationinformation related to one or more network communication protocols,including application protocols, secure communication protocols,client-server protocols, peer-to-peer protocols, shared file systemprotocols, protocol state machines, or the like, that may be employedfor protocol analysis, entity discovery, anomaly detection, or the like,in a monitored network environment.

Applications 320 may include computer executable instructions which,when executed by network computer 300, transmit, receive, or otherwiseprocess messages (e.g., SMS, Multimedia Messaging Service (MMS), InstantMessage (IM), email, or other messages), audio, video, and enabletelecommunication with another user of another mobile computer. Otherexamples of application programs include calendars, search programs,email client applications, IM applications, SMS applications, Voice OverInternet Protocol (VOIP) applications, contact managers, task managers,transcoders, database programs, word processing programs, securityapplications, spreadsheet programs, games, search programs, and soforth. Applications 320 may include network monitoring engine 322,inference engine 324, analysis engine 326, anomaly engine 327, webservices 329, or the like, that may be arranged to perform actions forembodiments described below. In one or more of the various embodiments,one or more of the applications may be implemented as modules orcomponents of another application. Further, in one or more of thevarious embodiments, applications may be implemented as operating systemextensions, modules, plugins, or the like.

Furthermore, in one or more of the various embodiments, networkmonitoring engine 322, inference engine 324, analysis engine 326,anomaly engine 327, web services 329, or the like, may be operative in acloud-based computing environment. In one or more of the variousembodiments, these applications, and others, that comprise themanagement platform may be executing within virtual machines or virtualservers that may be managed in a cloud-based based computingenvironment. In one or more of the various embodiments, in this contextthe applications may flow from one physical network computer within thecloud-based environment to another depending on performance and scalingconsiderations automatically managed by the cloud computing environment.Likewise, in one or more of the various embodiments, virtual machines orvirtual servers dedicated to network monitoring engine 322, inferenceengine 324, analysis engine 326, anomaly engine 327, web services 329,or the like, may be provisioned and de-commissioned automatically.

Also, in one or more of the various embodiments, network monitoringengine 322, inference engine 324, analysis engine 326, anomaly engine327, web services 329, or the like, may be located in virtual serversrunning in a cloud-based computing environment rather than being tied toone or more specific physical network computers. Likewise, in someembodiments, one or more of network monitoring engine 322, inferenceengine 324, analysis engine 326, anomaly engine 327, web services 329,or the like, may be configured to execute in a container-basedenvironment.

Further, network computer 300 may also comprise hardware security module(HSM) 360 for providing additional tamper resistant safeguards forgenerating, storing or using security/cryptographic information such as,keys, digital certificates, passwords, passphrases, two-factorauthentication information, or the like. In some embodiments, hardwaresecurity module may be employ to support one or more standard public keyinfrastructures (PKI), and may be employed to generate, manage, or storekeys pairs, or the like. In some embodiments, HSM 360 may be astand-alone network computer, in other cases, HSM 360 may be arranged asa hardware card that may be installed in a network computer.

Additionally, in one or more embodiments (not shown in the figures),network computer 300 may include one or more embedded logic hardwaredevices instead of CPUs, such as, an Application Specific IntegratedCircuit (ASIC), Field Programmable Gate Array (FPGA), Programmable ArrayLogic (PAL), or the like, or combination thereof. The embedded logichardware device may directly execute its embedded logic to performactions. Also, in one or more embodiments (not shown in the figures),the network computer may include one or more hardware microcontrollersinstead of CPUs. In one or more embodiments, the one or moremicrocontrollers may directly execute their own embedded logic toperform actions and access their own internal memory and their ownexternal Input and Output Interfaces (e.g., hardware pins or wirelesstransceivers) to perform actions, such as System On a Chip (SOC), or thelike.

Illustrative Logical System Architecture

FIG. 4 illustrates a logical architecture of system 400 for managingincident response operations based on network activity in accordancewith one or more of the various embodiments. System 400 may be arrangedto include a plurality of network devices or network computers on firstnetwork 402 and a plurality of network devices or network computers onsecond network 404. Communication between the first network and thesecond network is managed by switch 406. Also, NMC 408 may be arrangedto passively monitor or record packets (network packets) that arecommunicated in network flows between network devices or networkcomputers on first network 402 and second network 404. For example, thecommunication of flows of packets between the Host B network computerand the Host A network computer are managed by switch 406 and NMC 408may be passively monitoring and recording some or all of the networktraffic comprising these flows.

NMC 408 may be arranged to receive network communication for monitoringthrough a variety of means including network taps, wireless receivers,port mirrors or directed tunnels from network switches, clients orservers including the endpoints themselves, virtual machine, cloudcomputing instances, other network infrastructure devices, or the like,or combination thereof. In at least some of the various embodiments, theNMC may receive a copy of each packet on a particular network segment orvirtual local area network (VLAN). Also, for at least some of thevarious embodiments, NMCs may receive these packet copies through a portmirror on a managed Ethernet switch, e.g., a Switched Port Analyzer(SPAN) port, or a Roving Analysis Port (RAP). Port mirroring enablesanalysis and debugging of network communications. Port mirroring can beperformed for inbound or outbound traffic (or both) on single ormultiple interfaces. For example, in some embodiments, NMCs may bearranged to receive electronic signals over or via a physical hardwaresensor that passively receives taps into the electronic signals thattravel over the physical wires of one or more networks.

In one or more of the various embodiments, NMCs may be arranged toemploy adaptive networking monitoring information including one or moredevice relation models that enable inference engines or analysis enginesto monitor or record actions associated with investigating anomaliesthat may be detected in the monitored networks. Also, in someembodiments, NMCs may be arranged to instantiate one or more networkmonitoring engines, one or more inference engines, one or more anomalyengines, or the like, to provide investigation models that may beemployed to managing incident response operations based on networkactivity.

FIG. 5 illustrates a logical schematic of system 500 for managingincident response operations based on network activity in accordancewith one or more of the various embodiments. In one or more of thevarious embodiments, an NMC, such as NMC 502 may be arranged to monitornetwork traffic in one or more networks, such as, network 504, network506, or network 508. In this example, network 504, network 506, ornetwork 508 may be considered similar to network 108 or network 110.Also, in some embodiments, one or more of network 504, network 506, ornetwork 508 may be considered cloud computing environments. Likewise, insome embodiments, one or more of network 504, network 506, or network508 may be considered remote data centers, local data centers, or thelike, or combination thereof.

In one or more of the various embodiments, NMCs, such as NMC 502 may bearranged to communicate with one or more capture agents, such as,capture agent 512, capture agent 514, or capture agent 514. In someembodiments, capture agents may be arranged to selectively capturenetwork traffic or collect network traffic metrics that may be providedto NMC 502 for additional analysis.

In one or more of the various embodiments, capture agents may be NMCsthat are distributed in various networks or cloud environments. Forexample, in some embodiments, a simplified system may include one ormore NMCs that also provide capture agent services. In some embodiments,capture agents may be NMCs arranged to instantiate one or more captureengines to perform one or more capture or collection actions. Similarly,in one or more of the various embodiments, one or more capture agentsmay be instantiated or hosted separately from one or more NMCs.

In one or more of the various embodiments, capture agents may beselectively installed such that may capture metrics for select portionsof the monitored networks. Also, in some embodiments, in networks thathave groups or clusters of the same or similar entities, capture agentsmay be selectively installed on one or more entities that may berepresentative of entire groups or clusters pf similar entities. Thus,in some embodiments, capture agents on the representative entities maycollect metrics or traffic that may be used to infer the metrics oractivity associated with similarly situated entities that do not includea capture agent.

Likewise, in one or more of the various embodiments, one or more captureagents may be installed or activated for a limited time period tocollect information that may be used to infer activity information aboutthe monitored networks. Accordingly, in one or more of the variousembodiments, these one or more capture agents may be removed orde-activated if sufficient activity information or network traffic hasbeen collected.

In one or more of the various embodiments, system 500 may include one ormore network entities, such as, entities 518, entities 520, or the like,that communicate in or over one or more of the monitored networks.Entities 518 and entities 520 are illustrated here as cloud environmentcompute instances (e.g., virtual machines), or the like. However, one ofordinary skill in the art will appreciate that entities may beconsidered to be various network computers, network appliances, routers,applications, services, or the like, subject to network monitoring byone or more NMCs. (See, FIG. 4, as well).

In this example, for one or more of the various embodiments, captureagents, such as capture agent 512 may be arranged capture networktraffic or network traffic metrics associated with one or more entities,such as, entities 518. Accordingly, in some embodiments, some or all ofthe information captured by capture agents may be provided to one ormore NMCs, such as, NMC 502 for additional analysis. Also, in one ormore of the various embodiments, capture agents or NMCs may be arrangedto selectively store network traffic in a captured data store, such as,captured data store 522.

FIG. 6 illustrates a logical representation of network 600 in accordancewith at least one of the various embodiments. In at least one of thevarious embodiments, network 602 represents a physical network and theentities in the network. In this example, network 602 includes, networkcomputers 604, client computers 606, network devices, such as, networkdevice 610, and other items, such as, Wi-Fi hotspot 608. One of ordinaryskill in the art will appreciate that networks may have many more ordifferent devices than shown in FIG. 6.

In at least one of the various embodiments, one or more networkmonitoring computers (NMCs) may be arranged to monitor networks, suchas, network 602. (See, FIG. 4). In at least one of the variousembodiments, NMCs may be arranged to generate one or more devicerelation models that represent the entities in a network. For example,device relation model 612 represents a device relation modelcorresponding to network 602. Accordingly, device relation model 612includes nodes that represent the various entities that may be active innetwork 602. For example, entities 614, may represent some of theentities that are operative in network 602. In some embodiments, theremay be more entities in model 612 than the number of actual computersand network devices present in network 602 since many networkcomputers/devices may host more than one entity. For example, in someembodiments, a single network computer may host a web server and adatabase server. Accordingly, in this example, three entities may beincluded in the device relation model, one for the web server, one forthe database server, and one for the network computer itself.

In this example, device relation model 612 shows nodes that correspondto entities absent any edges. In some embodiments, initially some or allof the relationships between the entities may be unknown to themonitoring NMC, so some or all of the edges may be unknown and thereforomitted from device relation model 612. Note, in at least one of thevarious embodiments, there may be pre-defined networkarchitecture/topology information that may be available to the NMC.Accordingly, in some embodiments, the NMC may be able to determine someof the relationships between entities before observing network traffic.

FIG. 7 illustrates a logical representation of a portion of devicerelation model 700 in accordance with at least one of the variousembodiments. In at least one of the various embodiments, device relationmodels may include nodes that represent entities and edges thatrepresent relationships between the entities. In some embodiments,entities may represent servers, clients, switches, routers, NMCs, loadbalancers, applications, services, or the like. For example, entity 702may be a server entity that has relationships with other servers, suchas, entity 704 and entity 706. Likewise, entity 708 may be a server orother service that has a relationship with entity 704, entity 706, andentity 702. Further, entity 704 and entity 710 may have a relationshipand client entities 712 may have direct relationships with entity 710.

In at least one of the various embodiments, NMCs may be arranged to usedevice relation model 700 to discover relationships between groups ofentities. For example, device relation model 700 may be used todetermine that entity 702, entity 704, 710, and client 712 may be in arelated group because they are all on the same path through the graph.

In one or more of the various embodiments, one or more device relationmodels may be generated to represent different dimensions or conceptsthat may relate the one or more entities included in a model. Forexample, one device relation model may represent general dependenciesamong entities while another device relation model may be arranged torepresent administration dependencies that show which entities may bearranged to administer other entities.

Also, in one or more of the various embodiments, one or more devicerelation models may be arranged to represent various actions one or moreanomaly analysts, or the like, may take while conducting aninvestigation of one or more anomalies. In some embodiments, one or moredevice relation models may be directed to different networks orsub-networks.

FIGS. 8A and 8B illustrate how a device relation model may evolve as theNMCs gather more information about the relationships between theentities in a network.

FIG. 8A illustrates a logical representation of device relation model800 showing naïve relationships between the entities in accordance withthe one or more embodiments. In at least one of the various embodiments,for example, a NMC may initially determine the entities in a network byobserving the network traffic to obtain the source/destination networkaddress fields in the network packets that flow through the network. Inat least one of the various embodiments, each unique network address mayrepresent a different entity in the network.

Likewise, in some embodiments, the NMC may be arranged to observeresponses to broadcast messages, or the like. Additionally, in someembodiments, the NMC may be provided other configuration information(e.g., information provided by a configuration management database) thatdefines some or all of the entities in the network.

In this example, for at least one of the various embodiments, the NMChas discovered/identified six entities in the network (entity 802through entity 812). Accordingly, in some embodiments, the NMC may bearranged to generate a device relation model, such as, device relationmodel 800 that represents the six discovered entities as nodes in thegraph. Likewise, in some embodiments, edges in device relation model 800may represent the initial relationships as determined by the NMC. Forexample, in the initial stages of monitoring a network the NMC may bearranged to determine relationships based on which entities are observedto be communicating with each other.

However, in at least one of the various embodiments, NMCs may bearranged to provide a device relation model that represents therelationships between the entities that go beyond simpleinterconnectivity. Initially, in some embodiments, the NMC may definethe initial relationships in the network based on which entitiescommunicate with each other. However, in at least one of the variousembodiments, as the NMC collects more information about the entities andtheir relationships to other entities, the NMC may modify devicerelation model 800 to reflect the deeper understanding of theserelationships.

FIG. 8B illustrates a logical representation of device relation model800 showing informed relationships between the entities in accordancewith the one or more embodiments. In at least one of the variousembodiments, after sufficient monitoring has occurred, the NMC may haveobserved enough network traffic to evaluate and weight the relationshipsof the entities in the network.

In at least one of the various embodiments, some of the initialrelationships may be determined to be incidental, spurious, or otherwiseunimportant. Accordingly, the NMC may be arranged to remove (orde-prioritize) edges from device relation model 800 that correspond tosuch relationships. For example, in at least one of the variousembodiments, entities (e.g., Windows network domain controllers) in anetwork may be arranged to periodically exchange messages with one ormore other entities for discovery/accountability purposes. Thus, in thisexample, some of the messaging observed by an NMC may be routine andotherwise not resulting from an interesting relationships between thesender and receiver.

In at least one of the various embodiments, NMC may be arranged toevaluate the communication between entities to attempt to determine thetype of relationships and the importance of the relationships.Accordingly, in at least one of the various embodiments, NMCs may bearranged to collected metrics associated with the various network flowsflowing the network to identify the flows that may be important.Likewise, in at least one of the various embodiments, NMC may bearranged discover and recognize the communication protocols used byentities in monitored networks. In some embodiments, the NMCs may bearranged to use the collected metrics and its understanding of thecommunication protocol to establish or prioritize relationships betweenthe entities in the networks.

In this example, for at least one of the various embodiments, devicerelation model 800 has been modified to include relationships determinedto be of importance. The nodes representing entities 802-812 are stillpresent in but some of the edges that represent relationships in thenetwork have been removed. For example, in FIG. 8A, device relationmodel 800 includes an edge between entity 804 and entity 812. In FIG.8B, device relation model 800 omits the edge between entity 804 andentity 812.

In at least one of the various embodiments, the remaining edges indevice relation model 800 represent relationships between the entitiesthat the NMC determined to be important for a given device relationmodel. Note, in at least one of the various embodiments, an NMC mayemploy a variety of metrics, conditions, heuristics, or the like, toidentify relationships that may be of interest. For example, an NMC maybe arranged to identify entities that represent certain applications onthe network, such as, database servers, database clients, email servers,email clients, or the like, by identifying the communication protocolsthat may be used by the particular applications. In other cases, the NMCmay determine an important relationship based on the number or rate ofpackets exchanged between one or more entities. Accordingly, the NMC maybe configured to prioritize relationships between entities that exchangea high volume of traffic.

In at least one of the various embodiments, the NMC may analyze observedtraffic to identify network packets that flow through particular pathsin the device relation model. In some embodiments, NMCs may be arrangedto trace or identify such paths connecting related entities by observingcommon data carried in the payloads or header fields of the networkpackets that are passed among entities in the network. For example, anNMC may be arranged to observe sequence numbers, session identifiers,HTTP cookies, query values, or the like, from all entities participatingin transactions on the network. In some embodiments, the NMC maycorrelate observed network packets that may be requests and responsesbased on the contents of the network packets and known information aboutthe operation of the underlying applications or protocols.

FIGS. 9A and 9B provide additional illustration of how a device relationmodel may evolve as the NMCs gather more information about therelationships between the entities in a network.

FIG. 9A illustrates a logical representation of device relation model900 showing relationships between the entities based on observed networkconnections in accordance with the one or more embodiments. In at leastone of the various embodiments, the NMC has provided device relationmodel 900 that represents the relationships between entity 902 throughentity 912. Here device relation model 900 shows relationships that maybe associated with actual network links (e.g., physical links or virtuallinks) between the entities in the network. For example, the edges indevice relation model 900 may correspond to network flows that have beenobserved in the network. In some embodiments, an NMC may readily deducethese types of connection relationships by examining thesource/destination fields in network packets observed in the network.Accordingly, in this example, entity 906 may have been observedexchanging data with entity 908 over the network.

FIG. 9B illustrates a logical representation of device relation model900 showing phantom edges that represent relationships between theentities in accordance with the one or more embodiments. In someembodiments, networks may include entities that have importantlogical/operational relationships even though they do not exchangenetwork packets directly with each other. Here, the NMC has discoveredrelationships between entity 902 and entity 908 even though they do notcommunicate directly with each other. Likewise, the NMC has discoveredrelationships between entity 904 and entity 912 even though they do notcommunicate directly with each other. Similarly, entity 908, entity 910,entity 912 have been found to be related even though there is no directnetwork link or direct communication between them.

In at least one of the various embodiments, the NMC may be arranged torepresent such relationships using phantom edges. Phantom edges mayrepresent relationships between entities that do not correspond todirect network links. For example, entity 902 and entity 904 may bedatabase clients and entity 908, entity 910, and entity 912 may bedatabase servers. In this example, entity 902 and entity 904 access thedatabase servers through entity 906. In this example, entity 906 may beproxy-based load balancer of some kind. Accordingly, in this examplethere is no direct network link between the database clients and thedatabase servers. Nor, as represented, do the database server entities(entity 908, entity 910, and entity 912) have direct connections to eachother.

But, in some embodiments, the NMC may determine that the three databaseserver entities (entity 908, entity 910, and entity 912) are relatedbecause they are each receiving communications from the same loadbalancer (entity 906). Likewise, the NMC may determine a relationshipbetween the database clients (entity 902 and entity 904) and thedatabase servers (entity 908, entity 910, and entity 912) by observingthe operation of the database transactions even though they do notcommunicate directly with each other.

FIG. 10 illustrates a logical architecture of network 1000 that includesentities in accordance with the one or more embodiments. In at least oneof the various embodiments, networks may include several (100s, 1000s,or more) computers or devices that may put network traffic on thenetwork. As described above, (See, FIG. 4 and FIG. 5) network monitoringcomputers (NMCs) may be arranged to passively monitor the networktraffic. In some embodiments, NMCs (not shown in FIG. 10) may havedirect access to the wire traffic of the network enabling NMCs to accessall of the network traffic in monitored networks.

In at least one of the various embodiments, the NMC may be arranged toidentify entities in the network. Entities may include applications,services, programs, processes, network devices, or the like, operatingin the monitored network. For example, individual entities may include,web clients, web servers, database clients, database servers, mobile appclients, payment processors, groupware clients, groupware services, orthe like. In some cases, multiple entities may co-exist on the samenetwork computer, or cloud compute instance.

In this example, client computer 1002 may be hosting web client 1004 andDNS client 1006. Further, server computer 1008 may be hosting web server1010, database client 1014, and DNS client 1021. Also, in this example:server computer 1016 may be arranged to host database server 1018 andauthorization client 1020; server computer 1022 may be arranged to hostauthorization server 1024; and server computer 1026 may be arranged toDNS server 1028.

In at least one of the various embodiments, some or all of theapplications on a computer may correspond to entities. Generally,applications, services, or the like, that communicate using the networkmay be identified as entities by an NMC. Accordingly, there may be morethan one entity per computer. Some server computers may host manyentities. Also, some server computers may be virtualized machineinstances executing in a virtualized environment, such as, a cloud-basedcomputing environment. Likewise, one or more servers may running incontainerized compute instances, or the like.

In at least one of the various embodiments, an individual process orprogram running on a network computer may perform more than one type ofoperation on the network. Accordingly, some processes or programs may berepresented as more than one entity. For example, a web serverapplication may have an embedded database client. Thus, in someembodiments, an individual web server application may contribute two ormore entities to the device relation model.

In at least one of the various embodiments, the NMC may be arranged tomonitor the network traffic to identify the entities and to determinetheir roles. In at least one of the various embodiments, the NMC maymonitor the communication protocols, payloads, ports, source/destinationaddresses, or the like, or combination thereof, to identify entities.

In at least one of the various embodiments, the NMC may be preloadedwith configuration information that it may use to identify entities andthe services/roles they may be performing in the network. For example,if an NMC observes a HTTP GET request coming from a computer, it maydetermine there is a web client entity running on the host. Likewise, ifthe NMC observes a HTTP 200 OK response originating from a computer itmay determine that there is a web server entity in the network.

In at least one of the various embodiments, the NMC may use some or allof the tuple information included in network traffic to distinguishbetween different entities in the network. Further, the NMC may bearranged to track the connections and network flows established betweenseparate entities by correlating the tuple information of the requestsand responses between the entities.

FIG. 11 illustrates a logical representation of a data structure fordevice relation model 1100 that includes entities in accordance with theone or more embodiments. In at least one of the various embodiments,network monitoring computers (NMCs) may be arranged generate devicerelation models, such as, device relation model 1100. In this example,device relation model 1100 represents the entities discovered network1000 shown in FIG. 10.

In at least one of the various embodiments, as described above, NMCs mayarrange device relation models to represent the relationship theentities have to each other rather than just modeling the networktopology. For example, entity 1106, entity 1110, and entity 1118 areeach related to the DNS system in network 1000. Therefore, in thisexample, for some embodiments, the NMC may arrange device relation model1100 such that all of the DNS related entities (entity 1106, entity1110, and entity 1118) are neighbors in the graph. Accordingly, in someembodiments, even though entity 1106 corresponds to DNS client 1006 onclient computer 1002, the NMC may group entity 1106 with the other DNSentities rather than put it next other entities in the same computer.

In at least one of the various embodiments, the NMC may be arranged togenerate device relation model 1100 based on the relationships that theentities have with each other. Accordingly, in some embodiments, theedges in the graph may be selected or prioritized (e.g., weighted) basedon the type or strength of the relationship. In at least one of thevarious embodiments, the metrics used for prioritizing the edges in adevice relation model may be selected/computed based on configurationinformation that includes rules, conditions, pattern matching, scripts,computer readable instructions, or the like. In some embodiments, NMCsmay be arranged to apply this configuration information to the observednetwork packets (e.g., headers, payloads, or the like) to identify andevaluate relationships.

In at least one of the various embodiments, in device relation model1100, the edge connecting entity 1104 and entity 1108 is depictedthicker to represent the close relationship the web server entity haswith the database client entity. This reflects that the web server maybe hosting a data centric web application that fetches data from adatabase when it receives HTTP requests from clients. Likewise, fordevice relation model 1100 the relationship between the database client(entity 1108) and the database server (entity 1112) is also a strongrelationship. Similarly, the relationship between the authorizationclient (entity 1114) and the authorization server (entity 1116) is astrong relationship.

Also, in this example, the client (entity 1102) and DNS client 1 (entity1106) have a strong relationship and it follows that DNS client 1(entity 1106) has a strong relationship with the DNS server (entity1118). However, DNS client 2 (entity 1110) has a weak relationship withthe DNS server (entity 1118). In this example, this may make sensebecause DNS client 1 (entity 1106) is often used by the client (entity1102) to send lookup requests to the DNS server. In contrast, in thisexample, DNS client 2 (entity 1110) is rarely used since it is runningon the server computer (server computer 1008 in FIG. 10) and it mayrarely issue name lookup requests.

In at least one of the various embodiments, the NMC may traverse devicerelation model 1100 to identify entities that may be closely relatedtogether and associate them into a group. For example, in someembodiments, in device relation model 1100, entity 1104, entity 1108,and entity 1112 may be grouped since they each have strong relationshipswith each other.

Accordingly, in at least one of the various embodiments, the NMC may bearranged to correlate error signals that may be associated with one ormore entities that are in the same group to determine that an anomalymay be occurring. Related error signals that may propagate through agroup of closely related entities may be recognized as a bigger problemthat rises to an actual anomaly.

In at least one of the various embodiments, the NMC may be arranged tohave configuration information, including, templates, patterns, protocolinformation, or the like, for identifying error signals in a group thatmay have correlations that indicate they indicate an anomaly.

In some embodiments, the NMC may be arranged to capture/monitor incomingand outgoing network traffic for entities in a monitored network. Also,the NMC may be arranged to employ various protocol analysis facilities,such as, state machines, mathematical models, or the like, to trackexpected/normal operations of different types of entities in a monitorednetwork. Accordingly, in at least one of the various embodiments, theNMC may monitor the state of operations for entities that are workingtogether. For example, a web client entity, such as, entity 1102, maymake an HTTP request to web server entity 1104, that in turn triggersthe web server entity 1104 to issue a database request to DB cliententity 1108 that in turn is provided database server entity 1112. Insome embodiments, the NMC may monitor the operation of each entity inthe group by observing the network traffic exchanged between theentities in a group. Thus, in some embodiments, if an error at databaseserver entity 1112 causes web client entity 1102 to drop its connectionbecause of a timeout (or the user cancel the request, or repeats thesame request before the response is sent), the NMC may be able tocorrelate the error at database server entity 1112 with the “timeout”error at web client entity 1102 to recognize what may be a seriousanomaly.

FIG. 12 represents a logical representation of system 1200 fortransforming monitored network traffic into anomaly profile objects(e.g., anomaly profiles) or investigation profile objects (e.g.,investigation profiles) in accordance with one or more of the variousembodiments. In one or more of the various embodiments, NMC 1202 may bearranged to monitor network traffic 1204. As described, in someembodiments, NMC 1202 may be arranged to provide various metricsassociated with monitored network traffic 1204.

In one or more of the various embodiments, an NMC may be arranged totransform one or more collected metrics into anomaly profiles suitablefor classifying or categorizing one or more anomalous conditions thatmay be detected in the monitored networks.

Accordingly, in one or more of the various embodiments, as describedabove, NMCs such as, NMC 1202 may be arranged to collect metrics,portions of the network traffic, traffic attributes, or the like, frommonitored network traffic and arrange them into anomaly profiles. In oneor more of the various embodiments, anomaly profiles may includecollections of one or more fields with values that may be based onnetwork traffic 1204 or metrics associated with network traffic 1202. Inone or more of the various embodiments, one or more of the metricsincluded in an anomaly profile may correspond to metrics collected bythe NMC. In other embodiments, one or more of the metrics included in ananomaly profile may be composed of two or more metrics. Also, in one ormore of the various embodiments, one or more metrics or features of ananomaly profile may be computed based on one or more observed metrics.

Further, in one or more of the various embodiments, metric valuesincluded in anomaly profiles may be normalized or fit to a common schemaas well as arithmetically normalized. Normalizing metric values to acommon schema may include bucketing values. For example, in someembodiments, observed metrics that have continuous values may be mappedto named buckets, such as high, medium, low, or the like.

In one or more of the various embodiments, NMCs may be arranged toexecute one or more ingestion rules to perform pre-processing, such as,the data normalization, that may be required to map observed (raw)metrics into anomaly profile values or features. In one or more of thevarious embodiments, one or more ingestion rules may be built-in to NMCswhile other ingestion rules may be provided via configurationinformation, plug-ins, rule based policies, user input, or the like.

In one or more of the various embodiments, one or more anomaly profilesmay be associated with computer readable instructions that enforce oneor more matching rules or filter rules. Accordingly, in one or more ofthe various embodiments, inference engines may be arranged to executethe one or more matching rules or filters to determine if monitorednetwork activity should be associated with an anomaly profile. In someembodiments, one or more matching rules may be comprised of patternmatching instructions, such as, regular expressions, or the like. Insome embodiments, one or more matching rules may be comprised one ormore compound or cascading rules or sub-rules for matching networkactivity to anomaly profiles. In some embodiments, the same networkactivity may match two or more anomaly profiles.

In one or more of the various embodiments, anomaly profiles may includeone or more features that may include threshold values. For example, atraffic flood anomaly profile may include a feature or condition that ismet by one or more metrics, such as, the number of connection attemptsper minute or second, exceeding a threshold value. Likewise, in someembodiments, anomaly profiles may be associated other features, such as,the occurrence of one or more known error responses, latency or otherwait times exceeding a timeout value, unexpected attempts to accessprotected/critical entities, users performing unexpected or abnormalactivity, or the like. Generally, in one or more of the variousembodiments, the one or more features that define an anomaly profile maybe comprised of one or more metric values, one or more network trafficpatterns, or other monitored signals.

In some embodiments, anomaly profiles may be tagged or labeled by theNMCs, users, investigators, or the like. In one or more of the variousembodiments, NMCs may be arranged to automatically tag or label anomalyprofiles with generated labels or tags based on underlying networktraffic associated with the anomaly profile. Likewise, in someembodiments, users, analysts, or investigators may be enabled to add ormodify anomaly profile labels or tags to improve reporting,visualizations, human readability, or the like.

In one or more of the various embodiments, anomaly profiles may beemployed by inference engines, analysis engines, anomaly engines, or thelike, for detecting the occurrence one or more anomalies in a monitorednetwork. Accordingly, in one or more of the various embodiments, metricsrelated to the occurrence of network traffic associated with anomalyprofiles may be collected or tracked. For example, the number of timesnetwork traffic associated with a particular anomaly profile may berecorded as metric.

Similarly, in one or more of the various embodiments, investigationprofiles may be determined based on monitoring the network trafficassociated with the activity that analysts or investigators perform inresponse to the occurrence of anomaly. In some embodiments,investigation profiles may be arranged to represent the some or all ofthe sequence of steps or actions an investigator takes during aninvestigation.

In some embodiments, during the course of an investigation, aninvestigator may perform actions associated with two or moreinvestigation profiles. For example, a first investigation profile mayinclude navigating to a web page that displays an error report and asecond investigation profile may include the network activity that maybe associated with the investigator remote logging into a servercomputer and view log files in a command console.

In one or more of the various embodiments, an investigation profile mayinclude one or more investigation profiles such that the investigationprofile may be considered a compound investigation profile that iscomprised of two or more investigation profiles. For example, referringto previous example, a compound investigation profile may include thefirst investigation profile and the second investigation profileexamples described above.

In one or more of the various embodiments, NMCs may be arranged to trackinvestigator activity related to the investigator action in theinvestigation workspace (e.g., applications for managing orinvestigating incidents or anomalies), such as, user-interfaceinteractions, viewing reports or visualizations, annotations or commentsadded by the investigator, or the like. These local activities may beassociated with the network activity associated with the investigatoractions. Accordingly, in one or more of the various embodiments, theinvestigation profiles may include a record of local actions and networkactivity associated with an investigation.

Similar to anomaly profiles, in some embodiments, investigation profilesmay be comprised of one or more features that may be associated with oneor more metrics or one or more portions of network traffic as describedabove for anomaly profiles.

FIG. 13 illustrates a logical schematic of system 1300 for managingincident response operations based on network activity in accordancewith one or more of the various embodiments. In this example, system1300 include one or more monitored networks represented by networkingenvironment 1302. In this example, for some embodiments, NMC 1304 may bearranged to monitor network activity that may occur in environment 1302.Accordingly, in this example, NMC 1304 may be arranged to monitornetwork activity that may be associated with network device 1306, servercomputer 1308, network computer 1310, desktop computer 1312, or thelike. Accordingly, in some embodiments, NMC 1304 may be arranged tomonitor network traffic or collect metrics associated with variousactivities or interactions that may be occur in environment 1302. Forbrevity and clarity, traces or network paths associated with activityassociated with the entities, computers, or devices in environment 1302are not shown here.

In this example, for some embodiments, anomalous network activity, suchas, anomaly 1314, may be detected by NMC 1304. Accordingly, in one ormore of the various embodiments, NMC 1304 may associate anomaly 1314with one or more anomaly profiles and provide a notification, such as,notification 1316, to workstation 1318 which in this example may beoperated by a user, such as, investigator 1320. In response to thenotification of the occurrence of anomaly 1314, investigator 1320 may beenabled to take various actions to investigation the anomaly. In thisexample, investigator 1320 may perform actions such as logging intoserver computer 1308 from workstation 1318. In this example, logginginto server computer 1308 may generate network traffic or networkactivity 1322. Then, in this example, as the investigation into anomaly1314 continues, investigator 1320 may take one or more other actions,such as, using an application on server computer 1310 or logging intodesktop computer 1312, and so on. At the conclusion the investigation,investigator 1320 may indicate that the investigation is completed. Insome embodiments, investigator 1320 may be enabled to record additionalnotes or remarks related to the investigation. Also, in someembodiments, investigator 1320 may be enabled to associate one or morestatuses (e.g., success, failure, resolved, unresolved, complete,ignored, or the like) with the investigation.

In one or more of the various embodiments, NMC 1304 may be arranged tomonitor the network activity associated with the actions performed byinvestigator 1320 during the investigation of anomaly 1314. Accordingly,in one or more of the various embodiments, NMC 1304 may generate aninvestigation profile that based on some or all of the actions performedby investigator 1320 during the investigation anomaly 1314.

In one or more of the various embodiments, NMCs, such as, NMC 1304 maybe arranged to associate investigation profiles with the anomalyprofiles associated with the anomalies that triggered the investigationin the first place. In some embodiments, this may include associatinginvestigation profiles that had good outcomes as well as investigationprofiles that may have had bad outcomes. Accordingly, in one or more ofthe various embodiments, if an anomaly occurs again, the NMC may provideinvestigation information to investigators that includes one or moreinvestigation profiles that were previously used during the pastinvestigations of the other anomalies associated with the same anomalyprofile other previously encountered anomalies. In some embodiments,this may include investigation profiles that may be associated withsuccessful investigations or unsuccessful investigations as indicated bystatus information or other metrics associated with the one or moreinvestigation profiles.

Generalized Operations

FIGS. 14-20 represent generalized operations for managing incidentresponse operations based on network activity in accordance with one ormore of the various embodiments. In one or more of the variousembodiments, processes 1400, 1500, 1600, 1700, 1800, 1900, and 2000described in conjunction with FIGS. 14-20 may be implemented by orexecuted by one or more processors on a single network computer (ornetwork monitoring computer), such as network computer 300 of FIG. 3. Inother embodiments, these processes, or portions thereof, may beimplemented by or executed on a plurality of network computers, such asnetwork computer 300 of FIG. 3. In yet other embodiments, theseprocesses, or portions thereof, may be implemented by or executed on oneor more virtualized computers, such as, those in a cloud-basedenvironment. However, embodiments are not so limited and variouscombinations of network computers, client computers, or the like may beutilized. Further, in one or more of the various embodiments, theprocesses described in conjunction with FIGS. 14-20 may be used formanaging incident response operations based on network activity based onnetwork behavior in accordance with at least one of the variousembodiments or architectures such as those described in conjunction withFIGS. 4-13. Further, in one or more of the various embodiments, some orall of the actions performed by processes 1400, 1500, 1600, 1700, 1800,1900, and 2000 may be executed in part by network monitoring engine 322,inference engine 324, analysis engine 326, anomaly engine 327, or thelike, running on one or more processors of one or more networkcomputers.

FIG. 14 illustrates an overview flowchart of process 1400 for managingincident response operations based on network activity in accordancewith one or more of the various embodiments. After a start block, atblock 1402, in one or more of the various embodiments, one or more NMCsmay be arranged to collect one or more metrics or other informationbased on monitoring the network traffic in the monitored networks. Asdescribed above, NMCs may be arranged to monitor the network trafficassociated with various entities in the monitored networks. In someembodiments, the NMCs may employ some or all of the informationcollected during monitoring to generate one or more device relationmodels, anomaly profiles, investigation profiles, investigation models,or the like.

At decision block 1404, in one or more of the various embodiments, ifone or more anomalies are detected by one or more NMCs, control may flowto block 1406; otherwise, control may loop back to block 1402. Asdescribed above, NMCs may be arranged to include one or more rules forevaluating one or more metrics to evaluate some or all of the monitorednetwork traffic to determine if an anomaly has occurred in the monitorednetworks.

At block 1406, in one or more of the various embodiments, the one ormore NMCs may be arranged to provide one or more notifications to one ormore investigators. In one or more of the various embodiments, NMCs maybe arranged to provide notification messages that include informationabout the anomaly to one or more investigators that may be responsiblefor investigating the anomaly. In some embodiments, the one or more NMCsmay be arranged to provide the notification messages to another servicethat may route the notification to a responsible investigator. In otherembodiments, the NMCs may provide a user interface or application thatprovides an investigator workspace that includes one or moreapplications that enable the investigator to access one or moredashboards, visualizations, reports, or the like, that may be needed toconduct an investigation of the anomaly.

At block 1408, in one or more of the various embodiments, optionally,the one or more NMCs may be arranged to provide investigationinformation that may include one or more investigation profiles to oneor more investigator workspaces. In some cases, an anomalies may beassociated with investigation information that may be provided to theinvestigator. In some embodiments, anomalies may be associated withanomaly profiles that may be associated with one or more investigationprofiles that include one or more recommended actions that aninvestigator may perform to conduct an investigation of the anomaly.

In one or more of the various embodiments, investigation profiles mayinclude one or more automatic actions such as instructions to displayone or more interactive visualizations or interactive reports that maybe associated with the anomaly profile associated with the detectedanomaly.

In some cases, the investigation information may include a hyperlink orother entry point included in an email, text message, or the like, thatthe investigator may employ to display or access information orvisualizations associated with one or more metrics collected by theNMCs.

In one or more of the various embodiments, the investigation informationmay include two or more investigation profiles that may be presented inrank order based on the various performance scores or popularity scoresthat may be associated with investigation profile. For example, if twoor more investigation profiles are provided, the investigation profilesmay be listed in order of popularity (based on past use by the same orother investigators) reflecting past success of the same or otherinvestigators.

In one or more of the various embodiments, investigators may be enabledto select or open one or more provided investigation profiles to exposeor display a list of recommend actions that may be performed to conductthe investigation of the anomalies.

This block is marked optional because in some cases investigationinformation that may be associated with the detected anomalies may beunavailable.

At block 1410, in one or more of the various embodiments, the one ormore NMCs may be arranged to monitor network activity associated withthe investigation of the detected anomalies. As the investigatorperforms various actions to conduct an investigation of the anomaly,NMCs may monitor or record this investigation activity. In someembodiments, the NMCs may monitor the interactions the investigator haswith the investigation workspace. For example, the NMCs may be arrangedto monitor and track how or where the investigator clicks within theinvestigator workspace applications. For example, in one or more of thevarious embodiments, the NMC may track the visualization or reports thatare accessed by the investigator during the investigation.

Also, in one or more of the various embodiments, NMCs may be arranged tomonitor the network traffic or network activity associated with theinvestigation of the anomaly. For example, if the investigator connectsto another computer in the network, executes queries on remotedatabases, browses to one or more web sites or web applications, or thelike, the NMCs may monitor these actions. In one or more of the variousembodiments, the network activity associated with the investigation maybe correlated with the local workspace activity.

At block 1412, in one or more of the various embodiments, optionally,the one or more NMCs may be arranged to update or modify theinvestigation information based on the investigation activity. In someembodiments, as the investigator performs actions, the NMCs obtains moreinformation about investigation being conducted. Accordingly, in someembodiments, this additional information may information the NMCs thatadditional information, including one or more additional or alternativeinvestigation profiles should be provided to the investigator. Forexample, as the investigator performs actions to investigate theanomaly, the NMCs may be enabled to provide one or more investigationprofiles that are more relevant to the anomaly being investigated.

Likewise, in one or more of the various embodiments, if the NMCsdetermines that the investigator is not following a recommendedinvestigation profile, it may determine that the current recommendationsare insufficient. Thus, the NMC may be arranged to provide additionalinvestigation profiles based on the current investigation activity.

This block is marked optional because in some cases the investigationactivity associated with the investigation of the anomalies may nottrigger the investigation information that has been provide to theinvestigators to be updated. For example, the investigator may befollowing the actions provided in the investigation information.

At decision block 1414, in one or more of the various embodiments, ifthe investigation is closed, control may flow to block 1416; otherwise,control may loop back to block 1410. At some point, the investigator maydetermine that the investigation is complete whether it was successfulor not. Accordingly, the investigator may be enabled to provide anindication to the NMCs that the investigation of the anomaly iscomplete. In some embodiments, the investigator may be invited toprovide status or outcome information that indicates the success orfailure of the investigation.

At block 1416, in one or more of the various embodiments, the one ormore NMCs may be arranged to update one or more investigationinformation data stores or databases based on the actions taken by theinvestigators, the investigation outcome, or the like. In one or more ofthe various embodiments, NMCs may be arranged to store informationrelated to the anomalies, investigation activity, recommendedinvestigation profiles, investigation outcomes, or the like. Thisinformation may be used to generate additional investigation profiles,improve existing investigation profiles, generate investigation models,or the like. Next, control may be returned to a calling process.

FIG. 15 illustrates a flowchart of process 1500 for providing anomalyprofiles based on network activity in accordance with one or more of thevarious embodiments. After a start block, at block 1502, in one or moreof the various embodiments, one or more NMCs may be arranged to collectone or more metrics or other information based on monitoring the networktraffic in the monitored networks. As described above, NMCs may bearranged to monitor the network traffic associated with various entitiesin the monitored networks.

At decision block 1504, in one or more of the various embodiments, ifthe one or more NMCs detect one or more anomalies, control may flow toblock 1506; otherwise, control may loop back to 1504. As describedabove, NMCs may be arranged to include one or more rules for evaluatingone or more metrics or network traffic to determine if an anomaly hasoccurred in the monitored networks.

At block 1506, in one or more of the various embodiments, the one ormore NMCs may be arranged to determine the anomaly traffic from themonitored network traffic. In one or more of the various embodiments, asdescribed above, not all network traffic monitored in the monitorednetworks may be of interest with respect to the anomaly or anomalyprofiles. Accordingly, in one or more of the various embodiments, one ormore NMCs may be arranged to filter some or all of the monitored networktraffic or otherwise select a portion of the network traffic in themonitored networks associated with network activities that may be ofinterest.

At block 1508, in one or more of the various embodiments, the one ormore NMCs may be arranged to determine one or more features from theanomaly traffic. In some embodiments, NMCs may be arranged to determineone or more metrics to associate with the network traffic. In one ormore of the various embodiments, the selected one or more metrics mayinclude one or more metrics collected by the NMC, such as, source,destination, network protocol, application protocol, bit rate, packetsize, response latency, or the like, or combination thereof.

In one or more of the various embodiments, the one or more featurevalues may be arranged into a data structures or records, such as,vectors, lists, arrays, graphs, or the like. In some embodiments, thefeature values may be normalized or modified. Likewise, in someembodiments, one or more of the feature values may be associated withdiscrete categories or otherwise bucketed. For example, in one or moreof the various embodiments, one or more features having continuousvalues may be mapped to discrete values, such as, high, medium, or low.

At decision block 1510, in one or more of the various embodiments, ifthe one or more features match an existing anomaly profile, control mayflow to block 1514; otherwise, control may flow to block 1512.

At block 1512, in one or more of the various embodiments, the one ormore NMCs may be arranged to generate a new anomaly profile that may bearranged to match the features of the detected anomaly.

At block 1514, in one or more of the various embodiments, the one ormore NMCs may be arranged to provide one or more anomaly profiles basedon the determined features. In one or more of the various embodiments,the features associated with the network traffic associated withanomalies may be compared or matched against features associated withthe one or more anomaly profiles. In some embodiments, the comparisonsmay include comparing one or more patterns or masks that correspond toone or more features of the network traffic associated with one or moreanomalies. In some embodiments, a single feature such as, a URLassociated with a HTTP request, may be sufficient to map network trafficto an anomaly profile.

For example, in some embodiments, one feature interest may include a URLpattern that includes wildcards or positional parameters that may matchidentifiers or other query values included in the URL. Next, control maybe returned to a calling process.

FIG. 16 illustrates a flowchart of process 1600 for providinginvestigation profiles based on network activity in accordance with oneor more of the various embodiments. After a start block, at block 1602,in one or more of the various embodiments, one or more NMCs may bearranged to collect one or more metrics or other information based onmonitoring the network traffic in the monitored networks. As describedabove, NMCs may be arranged to monitor the network traffic associatedwith various entities in the monitored networks.

At block 1604, in one or more of the various embodiments, the one ormore NMCs may be arranged to determine investigation traffic from themonitored network traffic. In one or more of the various embodiments, asdescribed above, not all network traffic monitored in the monitorednetworks may be of interest with respect to investigation profiles.Accordingly, in one or more of the various embodiments, one or more NMCsmay be arranged to filter some or all of the monitored network trafficor otherwise select a portion of the network traffic in the monitorednetworks that may be associated with network activities that may be ofinterest.

At block 1606, in one or more of the various embodiments, the one ormore NMCs may be arranged to determine one or more features from theinvestigation traffic. In some embodiments, NMCs may be arranged todetermine one or more metrics to associate with the network traffic. Inone or more of the various embodiments, the selected one or more metricsmay include one or more metrics collected by the NMC, such as, source,destination, network protocol, application protocol, bit rate, packetsize, response latency, or the like, or combination thereof.

In one or more of the various embodiments, the one or more featurevalues may be arranged into a data structures or records, such as,vectors, lists, arrays, graphs, or the like. In some embodiments, thefeature values may be normalized or modified. Likewise, in someembodiments, one or more of the feature values may be associated withdiscrete categories or otherwise bucketed. For example, in one or moreof the various embodiments, one or more features having continuousvalues may be mapped to discrete values, such as, high, medium, or low.

At decision block 1608, in one or more of the various embodiments, ifthe one or more features match an existing investigation profile,control may flow to block 1612; otherwise, control may flow to block1610.

At block 1610, in one or more of the various embodiments, the one ormore NMCs may be arranged to generate a new investigation profile thatmay be arranged to match the determined features that may be associatedwith the investigation.

At block 1612, in one or more of the various embodiments, the one ormore NMCs may be arranged to provide one or more investigation profilesbased on the determined features. In one or more of the variousembodiments, the features associated with the network traffic associatedwith investigation activity performed by investigators investigating oneor more anomalies may be compared or matched against features associatedwith one or more investigation profiles. In some embodiments, thecomparisons may include comparing one or more patterns or masks thatcorrespond to one or more features of the network traffic associatedwith one or more investigation activities. In some embodiments, a singlefeature such as, a URL associated with a HTTP request, may be sufficientto map network traffic to an investigation profile.

For example, in some embodiments, one feature interest may include a URLpattern that includes wildcards or positional parameters that may matchidentifiers or other query values included in the URL. Next, control maybe returned to a calling process.

FIG. 17 illustrates a flowchart of process 1700 for managing incidentresponse operations based on network activity using anomaly profiles andinvestigation profiles in accordance with one or more of the variousembodiments. After a start block, at block 1702, in one or more of thevarious embodiments, one or more NMCs may be arranged to collect one ormore metrics or other information based on monitoring the networktraffic in the monitored networks. As described above, NMCs may bearranged to monitor the network traffic associated with various entitiesin the monitored networks.

At decision block 1704, in one or more of the various embodiments, ifthe one or more NMCs determines the one or more portions of the networktraffic matches or is associated with one or more anomaly profiles,control may flow block 1706; otherwise, control may loop back to block1702. As described above, in some embodiments, NMCs may be arranged tomatch or map one or more portions of the network traffic, one or moremetrics, or the like, to one or more features that may be used todetermine an anomaly profile to associate with the anomaly.

At block 1706, in one or more of the various embodiments, the one ormore NMCs may be arranged to determine one or more investigationprofiles that may be associated with the one or more anomaly profiles.As described above, NMCs may build up a catalog or database ofinvestigation profiles that may be associated with one or more anomalyprofiles. In some embodiments, one or more investigation models may beemployed to determine the one or more investigation profiles. In somecase, the NMCs may have previously associated one or more investigationprofiles with one or more anomaly profiles.

At decision block 1708, in one or more of the various embodiments, ifthere are one or more associated investigation profiles, control mayflow to block 1710; otherwise, control may flow to block 1712. In someembodiments, some anomalies or anomaly profiles may be unassociated withany investigation profiles. For example, if a never before detectedanomaly occurs, it may not be associated with an investigation profile.

At block 1710, in one or more of the various embodiments, the one ormore NMCs may be arranged to provide the one or more investigationprofiles to one or more investigators. As described above, investigationinformation that includes one or more investigation profiles may beprovided to an investigator.

At block 1712, in one or more of the various embodiments, the one ormore NMCs may be arranged to collect one or more metrics based onmonitored investigation activity. As the investigator performs one ormore actions to investigate the anomaly, the NMCs may collectinformation about the investigation activity and associated with theanomaly profile or investigation profiles (if any). In some embodiments,if there are no associated investigation profiles, the investigationactivity may be used to generate a new investigation profile that may beassociated with anomaly profile. Accordingly, if the same anomaly isdetected in the future, the investigation profile may be provided to theinvestigator to guide the investigation of the anomaly or otheranomalies that are associated with the same anomaly profile. Next,control may be returned to a calling process.

FIG. 18 illustrates a flowchart of process 1800 for managing incidentresponse operations based on network activity in accordance with one ormore of the various embodiments. After a start block, at block 1802, inone or more of the various embodiments, one or more NMCs may be arrangedto collect one or more metrics or other information based on monitoringthe network traffic in the monitored networks. As described above, NMCsmay be arranged to monitor the network traffic associated with variousentities in the monitored networks.

At decision block 1804, in one or more of the various embodiments, ifthe one or more NMCs determines the one or more portions of the networktraffic matches or is associated with one or more anomaly profiles,control may flow block 1806; otherwise, control may loop back to block1902.

At block 1806, in one or more of the various embodiments, the one ormore NMCs may be arranged to provide one or more investigation playbooksto one or more investigators. In one or more of the various embodiments,organizations may provide of define investigation playbooks that definespecific investigation activity that an investigator should performduring the investigation of some or all anomalies. In some embodiments,some anomalies may have specific investigation playbooks. In othercases, one or more general investigation playbooks may be provided.

In one or more of the various embodiments, investigation playbooks mayinclude one or more checklists, workflows, instructions, or the like,that an investigator may be expected to follow to investigate an anomalyor class of anomalies. In some embodiments, investigation playbooks maybe associated with one or more anomaly profiles. Accordingly, in one ormore of the various embodiments, if the detected anomaly is associatedwith an anomaly profile that is associated with an investigationplaybook, the NMCs may provide it to the investigator. In someembodiments, the investigator may already have access to one or moreinvestigation playbooks. Accordingly, in some embodiments, theinvestigator (or the investigation workspace) may provide theinvestigation playbook rather than the NMC.

At block 1808, in one or more of the various embodiments, the one ormore NMCs may be arranged to provide one or more investigation profilesto the one or more investigators. As described above, one or moreinvestigation profiles may be associated with the anomaly profile thatmay be associated with anomaly. Accordingly, if there are relevant(matching) investigation profiles available, the NMC may provide them tothe investigator.

At block 1810, in one or more of the various embodiments, the one ormore NMCs may be arranged to monitor the network traffic that may beassociated with the investigation activity performed by the one or moreinvestigators. In one or more of the various embodiments, investigatorsmay take various actions to investigate the anomalies. In someembodiments, the investigation profiles or investigation playbooks mayprovide instructions, checklists, workflows, or the like, that aninvestigator may use to guide them in the investigation of the anomaly.However, in one or more of the various embodiments, investigators may beenabled to perform investigation activities outside of the activitiesdefined in the investigation profiles or investigation playbooks.

Accordingly, as described above, the NMCs may monitor the investigationactivity performed by the investigator, including the activitiesprescribed by investigation profiles or investigation playbooks andactivity not included in investigation profiles or investigationplaybooks.

At block 1812, in one or more of the various embodiments, the one ormore NMCs may be arranged to compare the investigation activity to theone or more investigation playbooks. In one or more of the variousembodiments, NMCs may be arranged to evaluate if the investigatorfollowed the guidance of the investigation profiles or the investigationplaybook. For example, if the investigation profile prescribed fouractions in a particular order, the NMCs may track if the investigatorperformed the four actions in the prescribed order. Likewise, if theinvestigator was using an investigation playbook, the NMCs may track ifthe investigator performed the actions included in the playbook.

In some embodiments, the actual investigation activity performed by theinvestigator may be compared to the activity the investigator may reporthaving performed. For example, if the investigator reports that adatabase backup was performed before debugging the database as part ofthe investigation of the anomaly, the investigation activity monitoredby the NMCs may confirm that the database backup was actually performed.

At block 1814, in one or more of the various embodiments, the one ormore NMCs may be arranged to provide one or more compliance or deviationreports based on the comparison of the investigation activity to theinvestigation playbooks. In one or more of the various embodiments,reports that contain information related to how closely the investigatorfollowed the investigation playbooks or how the investigator followedthe investigation profiles may be determined based on the monitorinvestigation activity. As described above, the NMCs may be monitor thenetwork traffic or network activity associated with each investigationactivity. Accordingly, additions or omissions may be noted in a report.For example, if prescribed investigation profile or investigationplaybook included ten ordered actions, the NMCs may grade or score theinvestigation based on how many of the ten actions were performed, thetime it took to complete the investigation, how many of other additionalactions were performed, or the like.

In some embodiments, the report information may indicate that theinvestigator is not performing some or all of the actions recommended orprescribed by investigation playbooks or investigation profiles.Likewise, the reports may indicate that an investigation playbook isincorrect or inadequate for the anomaly that was being investigated.Accordingly, in some embodiments, organizations may determine if theircurrent investigation playbooks are sufficient or whether they need tobe updating. For example, in some embodiments, a low scoringinvestigation may represent a poorly designed investigation playbookrather than a poorly performing investigator.

Likewise, in some embodiments, if investigators deviate frominvestigation profiles this may indicate that the investigation profilesmay incorrect for the particular anomaly that was investigated. In thiscase, it may indicate that the investigation profile should bere-trained, re-optimized, discarded, or the like. Alternatively, in someembodiments, if an investigation model was employed to select theinvestigation profile, the deviation may be used to evaluate whether theinvestigation model is selecting relevant investigation profiles orwhether the investigation model requires re-training. Next, control maybe returned to a calling process.

FIG. 19 illustrates a flowchart of process 1900 for training oroptimizing improved investigation profiles based on historical anomalyprofile activity, historical investigation profile activity, andhistorical network activity in accordance with one or more of thevarious embodiments. In some embodiments, because some investigationprofiles may be generated based on actual network activity performed byinvestigators they may include redundant actions, risky actions,unnecessary actions, or the like, that may be introduced byinvestigators that are unfamiliar with some types of anomalies. Forexample, the first time an anomaly is encountered, investigators mayperform one or more exploratory actions that may ultimately beunnecessary to successfully the anomaly. Thus, in some cases, for someembodiments, one or more investigation profiles, especially newinvestigation profiles, may include actions that could be removed,re-ordered, combined, or the like. Accordingly, in one or more of thevarious embodiments, inference engines may be arranged to perform one ormore actions to attempt to optimize investigation profiles.

After a start block, at block 1902, in one or more of the variousembodiments, the one or more NMCs may be arranged to provide anomalyprofile history. In one or more of the various embodiments, NMCs may bearranged to track the amount of times that network activity associatedwith a given anomaly profile has been observed in the one or moremonitored networks. In some embodiments, this information may include adata sketch of the network activity that was associated with eachanomaly profile. In some embodiments, this may include one or moremetrics, such as, time of occurrence, entities in the network associatedwith the occurrence, duration of the activity, statistical valuesassociated with various metrics (e.g., mean, median, distributions orthe like), or the like.

In one or more of the various embodiments, NMCs may be provided (or maycapture) network traffic associated with the one or more anomalyprofiles. In some embodiments, the captured network traffic may bestored or indexed in other traffic/packet capture data stores.Accordingly, in one or more of the various embodiments, the NMCs mayrequest captured network traffic associated with the one or more anomalyprofile. For example, in one or more of the various embodiments, ananomaly profile may be associated with one or more incoming/outgoingnetwork messages, applications, services, ports, protocols, packetheader values, packet payload values, or the like, or combinationthereof. Accordingly, in this example, NMCs may provide a request (e.g.,provide a query) that includes one or more parameter values to selectcaptured network traffic or other historical metrics that may beassociated with one or more anomaly profiles.

At block 1904, in one or more of the various embodiments, the one ormore NMCs may be arranged to provide investigation profile history. Inone or more of the various embodiments, NMCs may be arranged to trackthe amount of times that network activity associated with a giveninvestigation profile has been observed in the one or more monitorednetworks. In some embodiments, this information may include a datasketch of the network activity that was associated with eachinvestigation profile. In some embodiments, this may include one or moremetrics, such as, time of occurrence, entities in the network associatedwith the occurrence, duration of the activity, statistical valuesassociated with various metrics (e.g., mean, median, distributions orthe like), or the like.

In one or more of the various embodiments, NMCs may be provided (or maycapture) captured network traffic associated with the one or moreinvestigation profiles. In some embodiments, the captured networktraffic may be stored or indexed in other traffic/packet capture datastores. Accordingly, in one or more of the various embodiments, the NMCsmay request captured network traffic associated with the one or moreinvestigation profiles. For example, in one or more of the variousembodiments, an investigation profile may be associated with one or moreincoming/outgoing network messages, applications, services, ports,protocols, packet header values, packet payload values, or the like, orcombination thereof. Accordingly, in this example, NMCs may provide arequest (e.g., provide a query) that includes one or more parametervalues to select captured network traffic or other historical metricsthat may be associated with one or more investigation profiles.

At block 1906, in one or more of the various embodiments, the one ormore NMCs may be arranged to evaluate one or more investigationprofiles. In one or more of the various embodiments, an inferenceengine, such as, inference engine 324, or the like, may be arranged toevaluate the one or more investigation profiles based on one or morefeatures of the provided anomaly profiles, investigation profiles, theanomaly profile history, or the investigation profile history.

In one or more of the various embodiments, the historical informationcollected above may include network activity information, includingmetrics collected by the NMCs, that show which investigation profileswere used to investigate various anomalies associated with the anomalyprofiles.

Accordingly, in one or more of the various embodiments, investigationprofiles may be evaluated based on the investigation success rates,average time to complete investigations, number of actions or steps perinvestigation, or the like.

Also, in one or more of the various embodiments, investigation profilesmay be evaluated based on risk factors or performance impacts associatedwith prescribed investigation actions. In some embodiments, each riskfactor or class of risk factors may be associated with a risk score.Likewise, in some embodiments, one or more actions may be associatedwith performance impact scores. Accordingly, in one or more of thevarious embodiments, investigation profiles may be associated with anaggregate risk score or aggregate performance score based on the scoreof the actions prescribed by a given investigation profile.

For example, in some embodiments, risky actions may include, accessingcritical entities, using commands that require super-user roles,starting/stopping critical processes, unsecure/unmonitored access ofsensitive data, moving or sharing sensitive information, decryptingnormally encrypted data, changing users, enabling remote access tosensitive entities, or the like.

Also, for example, in some embodiments, performance impacting actionsmay include actions that degrade host or network performance, such as,bulk copying, database dumps, backups, restoring from backups, bruteforce text searches, ad-hoc queries into production databases, queriesor searches of unstructured data stores, accessing remote entities orsystems having limited bandwidth or computing power, or the like.

In one or more of the various embodiments, NMCs may assign risk orperformance impact scores to various actions based on configurationinformation. Accordingly, organizations may be enabled to adjust riskscores or performance impact scores to various actions based on theiroperational requirements.

At block 1908, in one or more of the various embodiments, the one ormore NMCs may be arranged to optimize one or more investigationprofiles. In one or more of the various embodiments, inference enginesmay be arranged to determine one or more optimizations to improveexisting investigation profiles. In some embodiments, inference enginesmay identify one or more redundant actions that may be eliminated. Also,in some embodiments, an investigation profile may include two or moredifferent actions that produce the same result. For example, aninvestigation profile may prescribe that an investigator log intocomputer A to review resource A and then log into computer B to review Beven though resource A and resource B may be reviewed directly from theinvestigators workstation. Accordingly, in this example, an inferenceengine may detect the unnecessary remote access to computers A and B andrecommend that the steps of logging into computer A and computer B beremoved from the investigation profile.

Likewise, for example, inference engines may be arranged to re-order oneor more actions to reduce the number actions required for aninvestigation. For example, an investigation profile may prescribe thatthe investigator log into computer A to review resource A and thenlogout and perform some intervening steps before logging back intocomputer A to review resource B. In this example, the inference enginemay identify that one logon to computer A may be eliminated if resourceA and resource B are reviewed the first time the investigator logs in tocomputer A.

In some embodiments, inference engines may evaluate one or moreinvestigation profiles to identify one or more activities (e.g.,investigator actions, steps, or the like) that may be factored out tosimplify or otherwise reduce the complexity of one or more investigationprofiles identified, such as, discovering functionally equivalentinvestigation profiles that are more efficient.

In one or more of the various embodiments, the one or more NMCs may bearranged to employ one or more heuristics it identify one or morecircumstances that may be suitable for optimizations. In someembodiments, these heuristics may be defined using computer readableinstructions that may be built-in, provided by configurationinformation, provided by plugins, or the like.

Also, in one or more of the various embodiments, NMCs may employ machinelearning to learn one or more investigation profiles that may performbetter for some anomaly profiles than the investigation profilestypically or previously used by investigators. In some embodiments,machine learning may identify that investigators may be usingsub-optimal investigation profiles to investigate a given set ofanomalies. In some cases, this may be because investigators arefollowing an investigation playbook that is sub-optimal. Or, in somecases, investigators may be choosing the investigation profiles out ofhabit and not recognizing subtle differences in anomalies that degradethe performance or efficacy of the investigation profiles they arefollowing.

Accordingly, in one or more of the various embodiments, inferenceengines may be provided various anomaly profile features as inputs tolearn the optimized set of investigation profile features that arelikely to produce successful investigations. In some embodiments, thismay leverage circumstances where there may be several investigationprofiles that are associated with the same anomaly profiles. In suchcases, the inference engine may employ machine learning to learn whichfeatures from the several investigation profile features may be optimalfor investigating a given anomaly. In some cases, inference engine maygenerate investigation profiles based on learning from the investigationprofiles that were generated by monitoring investigation activityperformance by investigators.

Also, in one or more of the various embodiments, inference engines maybe arranged to employ machine learning to identify groupings ofanomalies, anomaly features, investigation profile features,investigation profile actions, or the like, that may be non-obvious.Accordingly, in one or more of the various embodiments, machine learningmay discover that several anomalies may have commonalities that may beinvestigated using the same steps or actions.

At block 1910, in one or more of the various embodiments, the one ormore NMCs may be arranged to deploy the one or more optimizedinvestigation profiles for use by investigators to investigate anomaliesthat may occur in the monitored networks. In one or more of the variousembodiments, the one or more optimized investigation profiles may bedeployed or activated for monitoring network traffic in the one or moremonitored networks. In some embodiments, users or investigators may beenabled to selectively activate or deactivate one or more investigationprofiles.

At block 1912, in one or more of the various embodiments, one or moreNMCs may be arranged to evaluate the performance of the one or moreinvestigation profiles. In one or more of the various embodiments,investigation profile performance may be monitored in real-time based onnetwork activity in the monitored networks.

In one or more of the various embodiments, investigation profiles may beevaluated based on their success or failure of the investigation ofanomalies. In some embodiments, investigation models may be associatedwith a score that represents the quality of investigation profilerecommendations. Accordingly, in some embodiments, poor evaluations mayresult in this performance score being decreased. Likewise, in someembodiments, good evaluations may result in this performance score beingincreased. Thus, in some embodiments, if the performance score of aninvestigation profile falls below a threshold value, the investigationmodel may be recommended for re-training or deactivation.

In some embodiments, the NMCs may be arranged to tag or flag the one ormore investigation profiles that produce poor results. In someembodiments, investigation profiles that have performance scores lessthan a defined threshold value may be automatically deactivated.

At block 1914, in one or more of the various embodiments, optionally,the one or more NMCs may be arranged to determine one or moreinvestigation profiles for re-optimization. In one or more of thevarious embodiments, NMCs may be arranged to periodically re-optimizesome or all investigation profiles. In some embodiments, allinvestigation profiles may be automatically selected for re-optimizationbased on performance scores.

In one or more of the various embodiments, the period forre-optimization investigation profiles may be impacted by other factors,such as, the network activity, anomaly profiles, entities, services,applications, sources, destinations, users, or the like, that may beassociated with an investigation profile. For example, one or moreinvestigation profiles associated with mission critical entities may beconfigured to be re-trained more often than investigation profilesassociated with less important entities. As these determinations may bedependent on the operational considerations of the monitored networks,NMCs may be arranged to employ configuration information provided byconfiguration files, file system policies, built-in defaults, userinput, or the like, combination thereof, to determine re-trainingfrequency or re-training sensitivity. Next, control may be returned to acalling process.

FIG. 20 illustrates a flowchart of process 2000 for providinginvestigation models based on anomaly profiles, investigation profiles,and network activity in accordance with one or more of the variousembodiments. After a start block, at block 2002, in one or more of thevarious embodiments, the one or more NMCs may be arranged to provideanomaly profile history as described above in the description for block1902.

At block 2004, in one or more of the various embodiments, the one ormore NMCs may be arranged to provide investigation profile history asdescribed above in the description of block 1904

At block 2006, in one or more of the various embodiments, the one ormore NMCs may be arranged to provide one or more candidate investigationmodels. In one or more of the various embodiments, an inference engine,such as, inference engine 324, or the like, may be arranged to generatethe one or more investigation models based on one or more features ofthe provided anomaly profiles, investigation profiles, the anomalyprofile history, or the investigation profile history. In one or more ofthe various embodiments, the investigation models may be arranged todetermine select one or more investigation profiles that may be usedinvestigate one or more anomalies.

In one or more of the various embodiments, investigation models may bearranged to select one or more investigation profiles given one or moreanomaly profiles or anomalies. Accordingly, in one or more of thevarious embodiments, one or more investigation models may include one ormore classifiers that may be generated or trained using one or moreconventional machine learning techniques.

In one or more of the various embodiments, the historical informationcollected above may include network activity information, includingmetrics collected by the NMCs, that show which investigation profileswere used to investigate various anomalies associated with the anomalyprofiles. In some embodiments, machine learning may be used to trainclassifiers that may be used recommend investigation profiles foranomalies that have not been previously encountered.

Also, in one or more of the various embodiments, NMCs may employ machinelearning to discover one or more investigation profiles that may performbetter for some anomaly profiles than the investigation profilestypically or previously used by investigators. In some embodiments,machine learning may identify that investigators may be usingsub-optimal investigation profiles to investigate a given set ofanomalies. In some cases, this may be because investigators arefollowing an investigation playbook that is sub-optimal. Or, in somecases, investigators may be choosing the investigation profiles out ofhabit and not recognizing subtle differences in anomalies that degradethe performance of the investigation profiles they are using.

In one or more of the various embodiments, classifiers trained bymachine learning may identify investigation profiles that have fewersteps or a highly likelihood of success than the investigation profilescommonly used to investigate a given anomaly.

At block 2008, in one or more of the various embodiments, the one ormore NMCs may be arranged to obtain feedback associated with the one ormore candidate investigation models. In one or more of the variousembodiments, NMCs may be arranged to provide interactive reports thatenable users or investigators to review the investigation models.Accordingly, in one or more of the various embodiments, users orinvestigators may be enabled score or rate one or more investigationmodels.

In one or more of the various embodiments, as described above,investigators may be enabled to submit an investigation status or resultwhen an investigation is closed. In some embodiments, the one or moreNMCs may be arranged to enable investigators provide additionalfeedback, such as, grades, ranks, or the like, that may be used toevaluate the investigation profiles that were recommended by aninvestigation model. Accordingly, in one or more of the variousembodiments, if an investigation model is recommending investigationprofiles that receive poor grades from investigators, the investigationmodel may be evaluated poorly.

At block 2010, in one or more of the various embodiments, the one ormore NMCs may be arranged to deploy the one or more investigation modelsfor use by investigators to investigate anomalies that may occur in themonitored networks. In one or more of the various embodiments, one ormore investigation models may be deployed or activated for monitoringnetwork traffic in the one or more monitored networks. In someembodiments, users or investigators may be enabled to selectivelyactivate or deactivate one or more investigation models.

At block 2012, in one or more of the various embodiments, one or moreNMCs may be arranged to evaluate the performance of the one or moreinvestigation models. In one or more of the various embodiments,investigation model performance may be monitored in real-time based onnetwork activity in the monitored networks.

In one or more of the various embodiments, investigation models may beevaluated based on the success or failure of the investigations based oninvestigation profiles that may be recommended by the investigationmodels. For example, in some embodiments, in response to the occurrenceof an anomaly, an investigation model may select one or moreinvestigation profiles to provide to an investigator. If theinvestigator follows the investigation profile and the investigation isunsuccessful, the investigation model may receive a poor evaluation. Insome embodiments, investigation models may be associated with a scorethat represents the quality of investigation profile recommendations.Accordingly, in some embodiments, poor evaluations may result in thisperformance score being decreased. Likewise, in some embodiments, goodevaluations may result in this performance score being increased. Thus,in some embodiments, if the performance score falls below a thresholdvalue, the investigation model may be recommended for re-training.

In some embodiments, the NMCs may be arranged to tag or flag the one ormore investigation models that receive poor evaluations. In someembodiments, investigation models that receive an evaluation score thatis less than a defined threshold value may be automatically deactivated.In some embodiments, this may be considered a temporary adjustment thatmay remain in effect until the investigation model is re-trained or theevaluation of the investigation model changes.

In one or more of the various embodiments, the one or more NMCs may bearranged to evaluate the performance of investigators based on if theyfollow the recommended investigation profiles. For example, in someembodiments, investigators that fail to follow recommended investigationprofiles and unsuccessfully investigate an anomaly may receive poorevaluations. Also, for example, investigators that perform additionalinvestigative actions or activity that resolve the investigation whenthe actions associated with the recommended investigation profiles fail,may receive good evaluations.

At block 2014, in one or more of the various embodiments, optionally,the one or more NMCs may be arranged to determine one or moreinvestigation models for re-training. In one or more of the variousembodiments, NMCs may be arranged to periodically re-train some or allinvestigation models. In some embodiments, all investigation models maybe automatically selected for re-training. In other embodiments, one ormore investigation models may be selected for re-training based onevaluation scores associated with the investigation models.

In one or more of the various embodiments, the period for re-traininginvestigation models may be impacted by other factors, includinginvestigation model priority, investigation model category, or the like.Accordingly, in one or more of the various embodiments, selecting aninvestigation model for re-training may depend on variouscharacteristics of the network activity, anomaly profiles, investigationprofiles, or the like, that may be associated with the investigationmodel, such as, entities, services, applications, sources, destinations,users, or the like, or combination thereof. For example, one or moreinvestigation models associated with mission critical entities may beconfigured to be re-trained more often than one or more investigationmodels that may be associated with less important entities. As thesedeterminations may be dependent on the operational considerations of themonitored networks, NMCs may be arranged to employ configurationinformation provided by configuration files, file system policies,built-in defaults, user input, or the like, combination thereof, todetermine re-training frequency or re-training sensitivity. Next,control may be returned to a calling process.

It will be understood that each block of the flowchart illustration, andcombinations of blocks in the flowchart illustration, can be implementedby computer program instructions. These program instructions may beprovided to a processor to produce a machine, such that theinstructions, which execute on the processor, create means forimplementing the actions specified in the flowchart block or blocks. Thecomputer program instructions may be executed by a processor to cause aseries of operational steps to be performed by the processor to producea computer-implemented process such that the instructions, which executeon the processor to provide steps for implementing the actions specifiedin the flowchart block or blocks. The computer program instructions mayalso cause at least some of the operational steps shown in the blocks ofthe flowchart to be performed in parallel. Moreover, some of the stepsmay also be performed across more than one processor, such as mightarise in a multi-processor computer system. In addition, one or moreblocks or combinations of blocks in the flowchart illustration may alsobe performed concurrently with other blocks or combinations of blocks,or even in a different sequence than illustrated without departing fromthe scope or spirit of the invention.

Accordingly, blocks of the flowchart illustration support combinationsof means for performing the specified actions, combinations of steps forperforming the specified actions and program instruction means forperforming the specified actions. It will also be understood that eachblock of the flowchart illustration, and combinations of blocks in theflowchart illustration, can be implemented by special purpose hardwarebased systems, which perform the specified actions or steps, orcombinations of special purpose hardware and computer instructions. Theforegoing example should not be construed as limiting or exhaustive, butrather, an illustrative use case to show an implementation of at leastone of the various embodiments of the invention.

Further, in one or more embodiments (not shown in the figures), thelogic in the illustrative flowcharts may be executed using an embeddedlogic hardware device instead of a CPU, such as, an Application SpecificIntegrated Circuit (ASIC), Field Programmable Gate Array (FPGA),Programmable Array Logic (PAL), or the like, or combination thereof. Theembedded logic hardware device may directly execute its embedded logicto perform actions. In one or more embodiment, a microcontroller may bearranged to directly execute its own embedded logic to perform actionsand access its own internal memory and its own external Input and OutputInterfaces (e.g., hardware pins or wireless transceivers) to performactions, such as System On a Chip (SOC), or the like.

1. A method for monitoring network traffic using one or more networkcomputers, wherein execution of instructions by the one or more networkcomputers perform the method comprising: instantiating a monitoringengine to perform actions, including: monitoring network trafficassociated with a plurality of entities in one or more networks toprovide one or more metrics; generating a device relation model forrepresenting direct and indirect relationships between the plurality ofentities; dynamically modifying the device relation model based on oneor more priorities of the one or more direct and indirect relationshipsto one or more of the plurality of entities, wherein the one or morepriorities are based on communication between the plurality of entitiesthat are employed to generate one or more of a type or a weight for theone or more direct and indirect relationships; and determining ananomaly based on the one or more metrics exceeding one or more thresholdvalues; and instantiating an inference engine that performs actions,including: providing an anomaly profile from a plurality of anomalyprofiles based on one or more portions of the network traffic that areassociated with the anomaly and the device relationship model; providingan investigation profile from a plurality of investigation profilesbased on the anomaly profile, wherein the investigation profile includesa representation of one or more of classes, types or categorizations andinformation associated with one or more previously performedinvestigation activities and results associated with one or moreprevious investigations of the anomaly; monitoring the investigation ofthe anomaly based on one or more other portions of the network traffic,wherein the one or more other portions of the network traffic areassociated with monitoring an occurrence of the one or moreinvestigation activities; and modifying a performance score that isassociated with the investigation profile based on the occurrence of theone or more investigation activities and a completion status of theinvestigation, wherein the performance score is decreased when one ormore other investigation activities are included in the investigation orwhen one or more of the one or more investigation activities are omittedfrom the investigation of the anomaly.
 2. The method of claim 1, whereinproviding the investigation profile, further comprises: providing one ormore investigation models that are trained to classify anomaly profiles;employing the one or more investigation models to classify the anomalyprofile; and providing the investigation profile based on aclassification of the anomaly profile.
 3. The method of claim 1, whereinthe inference engine performs further actions, including: providingnetwork activity information that is associated with one or moreprevious occurrences of one or more anomalies; providing investigationactivity information that is associated with previous investigationsassociated with one or more investigation profiles and the one or moreanomalies; and evaluating the one or more investigation profiles basedon the investigation activity information associated with previousinvestigations of the one or more anomalies, wherein the one or moreinvestigation profiles are optimized based on the evaluation.
 4. Themethod of claim 1, wherein the inference engine performs furtheractions, including: providing network activity information that isassociated with one or more previous occurrences of one or moreanomalies; providing investigation activity information and completionresults that are associated with previous investigations associated withone or more investigation profiles and the one or more anomalies;training one or more investigation models to provide an investigationprofile based on an input that includes an input anomaly profile,investigation activity information, and completion results; andre-training the one or more investigation models when a confidence scoreassociated with the one or more investigation models is less than athreshold value.
 5. The method of claim 1, wherein the inference engineperforms further actions, including: providing a playbook that definesone or more actions for investigating the anomaly; comparing theoccurrence of the one or more investigation activities with the one ormore actions defined in the playbook to provide a deviation score,wherein the deviation score is associated with a number of the one ormore actions that are not performed during the investigation; andevaluating an efficacy of the playbook based on the deviation score andthe completion result associated with the investigation.
 6. The methodof claim 1, wherein providing the anomaly profile further comprises:providing one or more features associated with the anomaly based on theone or more portions of the network traffic that are associated with theanomaly; comparing the one or more features to one or more otherfeatures that are associated with the plurality of anomaly profiles; andgenerating the anomaly profile based on a negative result of thecomparison, wherein the anomaly profile is generated based on the one ormore features.
 7. The method of claim 1, wherein the inference engineperforms further actions, including, generating the investigationprofile based on the one or more other portions of the network trafficthat are associated with the one or more investigation activities. 8.The method of claim 1, wherein the inference engine performs furtheractions, including, providing one or more reports based on theinvestigation and the completion result, wherein the one or more reportsare displayed to one or more users.
 9. A processor readablenon-transitory storage media that includes instructions for monitoringnetwork traffic using one or more network monitoring computers, whereinexecution of the instructions by the one or more network computersperform the method comprising: instantiating a monitoring engine toperform actions, including: monitoring network traffic associated with aplurality of entities in one or more networks to provide one or moremetrics; generating a device relation model for representing direct andindirect relationships between the plurality of entities; dynamicallymodifying the device relation model based on one or more priorities ofthe one or more direct and indirect relationships to one or more of theplurality of entities, wherein the one or more priorities are based oncommunication between the plurality of entities that are employed togenerate one or more of a type or a weight for the one or more directand indirect relationships; and determining an anomaly based on the oneor more metrics exceeding one or more threshold values; andinstantiating an inference engine that performs actions, including:providing an anomaly profile from a plurality of anomaly profiles basedon one or more portions of the network traffic that are associated withthe anomaly and the device relationship model; providing aninvestigation profile from a plurality of investigation profiles basedon the anomaly profile, wherein the investigation profile includes arepresentation of one or more of classes, types or categorizations andinformation associated with one or more previously performedinvestigation activities and results associated with one or moreprevious investigations of the anomaly; monitoring the investigation ofthe anomaly based on one or more other portions of the network traffic,wherein the one or more other portions of the network traffic areassociated with monitoring an occurrence of the one or moreinvestigation activities; and modifying a performance score that isassociated with the investigation profile based on the occurrence of theone or more investigation activities and a completion status of theinvestigation, wherein the performance score is decreased when one ormore other investigation activities are included in the investigation orwhen one or more of the one or more investigation activities are omittedfrom the investigation of the anomaly.
 10. The media of claim 9, whereinproviding the investigation profile, further comprises: providing one ormore investigation models that are trained to classify anomaly profiles;employing the one or more investigation models to classify the anomalyprofile; and providing the investigation profile based on aclassification of the anomaly profile.
 11. The media of claim 9, whereinthe inference engine performs further actions, including: providingnetwork activity information that is associated with one or moreprevious occurrences of one or more anomalies; providing investigationactivity information that is associated with previous investigationsassociated with one or more investigation profiles and the one or moreanomalies; and evaluating the one or more investigation profiles basedon the investigation activity information associated with previousinvestigations of the one or more anomalies, wherein the one or moreinvestigation profiles are optimized based on the evaluation.
 12. Themedia of claim 9, wherein the inference engine performs further actions,including: providing network activity information that is associatedwith one or more previous occurrences of one or more anomalies;providing investigation activity information and completion results thatare associated with previous investigations associated with one or moreinvestigation profiles and the one or more anomalies; training one ormore investigation models to provide an investigation profile based onan input that includes an input anomaly profile, investigation activityinformation, and completion results; and re-training the one or moreinvestigation models when a confidence score associated with the one ormore investigation models is less than a threshold value.
 13. The mediaof claim 9, wherein the inference engine performs further actions,including: providing a playbook that defines one or more actions forinvestigating the anomaly; comparing the occurrence of the one or moreinvestigation activities with the one or more actions defined in theplaybook to provide a deviation score, wherein the deviation score isassociated with a number of the one or more actions that are notperformed during the investigation; and evaluating an efficacy of theplaybook based on the deviation score and the completion resultassociated with the investigation.
 14. The media of claim 9, whereinproviding the anomaly profile further comprises: providing one or morefeatures associated with the anomaly based on the one or more portionsof the network traffic that are associated with the anomaly; comparingthe one or more features to one or more other features that areassociated with the plurality of anomaly profiles; and generating theanomaly profile based on a negative result of the comparison, whereinthe anomaly profile is generated based on the one or more features. 15.The media of claim 9, wherein the inference engine performs furtheractions, including, generating the investigation profile based on theone or more other portions of the network traffic that are associatedwith the one or more investigation activities.
 16. The media of claim 9,wherein the inference engine performs further actions, including,providing one or more reports based on the investigation and thecompletion result, wherein the one or more reports are displayed to oneor more users.
 17. A system for monitoring network traffic in a network:one or more network computers, comprising: a transceiver thatcommunicates over the network; a memory that stores at leastinstructions; and one or more processors that execute instructions thatperform actions, including: instantiating a monitoring engine to performactions, including: monitoring network traffic associated with aplurality of entities in one or more networks to provide one or moremetrics; generating a device relation model for representing direct andindirect relationships between the plurality of entities; dynamicallymodifying the device relation model based on one or more priorities ofthe one or more direct and indirect relationships to one or more of theplurality of entities, wherein the one or more priorities are based oncommunication between the plurality of entities that are employed togenerate one or more of a type or a weight for the one or more directand indirect relationships; and determining an anomaly based on the oneor more metrics exceeding one or more threshold values; andinstantiating an inference engine that performs actions, including:providing an anomaly profile from a plurality of anomaly profiles basedon one or more portions of the network traffic that are associated withthe anomaly and the device relationship model; providing aninvestigation profile from a plurality of investigation profiles basedon the anomaly profile, wherein the investigation profile includes arepresentation of one or more of classes, types or categorizations andinformation associated with one or more previously performedinvestigation activities and results associated with one or moreprevious investigations of the anomaly; monitoring the investigation ofthe anomaly based on one or more other portions of the network traffic,wherein the one or more other portions of the network traffic areassociated with monitoring an occurrence of the one or moreinvestigation activities; and modifying a performance score that isassociated with the investigation profile based on the occurrence of theone or more investigation activities and a completion status of theinvestigation, wherein the performance score is decreased when one ormore other investigation activities are included in the investigation orwhen one or more of the one or more investigation activities are omittedfrom the investigation of the anomaly; and one or more client computers,comprising: a transceiver that communicates over the network; a memorythat stores at least instructions; and one or more processors thatexecute instructions that perform actions, including: providing one ormore of the one or more portions of the network traffic.
 18. The systemof claim 17, wherein providing the investigation profile, furthercomprises: providing one or more investigation models that are trainedto classify anomaly profiles; employing the one or more investigationmodels to classify the anomaly profile; and providing the investigationprofile based on a classification of the anomaly profile.
 19. The systemof claim 17, wherein the inference engine performs further actions,including: providing network activity information that is associatedwith one or more previous occurrences of one or more anomalies;providing investigation activity information that is associated withprevious investigations associated with one or more investigationprofiles and the one or more anomalies; and evaluating the one or moreinvestigation profiles based on the investigation activity informationassociated with previous investigations of the one or more anomalies,wherein the one or more investigation profiles are optimized based onthe evaluation.
 20. The system of claim 17, wherein the inference engineperforms further actions, including: providing network activityinformation that is associated with one or more previous occurrences ofone or more anomalies; providing investigation activity information andcompletion results that are associated with previous investigationsassociated with one or more investigation profiles and the one or moreanomalies; training one or more investigation models to provide aninvestigation profile based on an input that includes an input anomalyprofile, investigation activity information, and completion results; andre-training the one or more investigation models when a confidence scoreassociated with the one or more investigation models is less than athreshold value.
 21. The system of claim 17, wherein the inferenceengine performs further actions, including: providing a playbook thatdefines one or more actions for investigating the anomaly; comparing theoccurrence of the one or more investigation activities with the one ormore actions defined in the playbook to provide a deviation score,wherein the deviation score is associated with a number of the one ormore actions that are not performed during the investigation; andevaluating an efficacy of the playbook based on the deviation score andthe completion result associated with the investigation.
 22. The systemof claim 17, wherein providing the anomaly profile further comprises:providing one or more features associated with the anomaly based on theone or more portions of the network traffic that are associated with theanomaly; comparing the one or more features to one or more otherfeatures that are associated with the plurality of anomaly profiles; andgenerating the anomaly profile based on a negative result of thecomparison, wherein the anomaly profile is generated based on the one ormore features.
 23. The system of claim 17, wherein the inference engineperforms further actions, including, generating the investigationprofile based on the one or more other portions of the network trafficthat are associated with the one or more investigation activities.
 24. Anetwork computer for monitoring communication over a network between twoor more computers, comprising: a transceiver that communicates over thenetwork; a memory that stores at least instructions; and one or moreprocessors that execute instructions that perform actions, including:instantiating a monitoring engine to perform actions, including:monitoring network traffic associated with a plurality of entities inone or more networks to provide one or more metrics; generating a devicerelation model for representing direct and indirect relationshipsbetween the plurality of entities; dynamically modifying the devicerelation model based on one or more priorities of the one or more directand indirect relationships to one or more of the plurality of entities,wherein the one or more priorities are based on communication betweenthe plurality of entities that are employed to generate one or more of atype or a weight for the one or more direct and indirect relationships;and determining an anomaly based on the one or more metrics exceedingone or more threshold values; and instantiating an inference engine thatperforms actions, including: providing an anomaly profile from aplurality of anomaly profiles based on one or more portions of thenetwork traffic that are associated with the anomaly and the devicerelationship model; providing an investigation profile from a pluralityof investigation profiles based on the anomaly profile, wherein theinvestigation profile includes a representation of one or more ofclasses, types or categorizations and information associated with one ormore previously performed investigation activities and resultsassociated with one or more previous investigations of the anomaly;monitoring the investigation of the anomaly based on one or more otherportions of the network traffic, wherein the one or more other portionsof the network traffic are associated with monitoring an occurrence ofthe one or more investigation activities; and modifying a performancescore that is associated with the investigation profile based on theoccurrence of the one or more investigation activities and a completionstatus of the investigation, wherein the performance score is decreasedwhen one or more other investigation activities are included in theinvestigation or when one or more of the one or more investigationactivities are omitted from the investigation of the anomaly.
 25. Thenetwork computer of claim 24, wherein providing the investigationprofile, further comprises: providing one or more investigation modelsthat are trained to classify anomaly profiles; employing the one or moreinvestigation models to classify the anomaly profile; and providing theinvestigation profile based on a classification of the anomaly profile.26. The network computer of claim 24, wherein the inference engineperforms further actions, including: providing network activityinformation that is associated with one or more previous occurrences ofone or more anomalies; providing investigation activity information thatis associated with previous investigations associated with one or moreinvestigation profiles and the one or more anomalies; and evaluating theone or more investigation profiles based on the investigation activityinformation associated with previous investigations of the one or moreanomalies, wherein the one or more investigation profiles are optimizedbased on the evaluation.
 27. The network computer of claim 24, whereinthe inference engine performs further actions, including: providingnetwork activity information that is associated with one or moreprevious occurrences of one or more anomalies; providing investigationactivity information and completion results that are associated withprevious investigations associated with one or more investigationprofiles and the one or more anomalies; training one or moreinvestigation models to provide an investigation profile based on aninput that includes an input anomaly profile, investigation activityinformation, and completion results; and re-training the one or moreinvestigation models when a confidence score associated with the one ormore investigation models is less than a threshold value.
 28. Thenetwork computer of claim 24, wherein the inference engine performsfurther actions, including: providing a playbook that defines one ormore actions for investigating the anomaly; comparing the occurrence ofthe one or more investigation activities with the one or more actionsdefined in the playbook to provide a deviation score, wherein thedeviation score is associated with a number of the one or more actionsthat are not performed during the investigation; and evaluating anefficacy of the playbook based on the deviation score and the completionresult associated with the investigation.
 29. The network computer ofclaim 24, wherein providing the anomaly profile further comprises:providing one or more features associated with the anomaly based on theone or more portions of the network traffic that are associated with theanomaly; comparing the one or more features to one or more otherfeatures that are associated with the plurality of anomaly profiles; andgenerating the anomaly profile based on a negative result of thecomparison, wherein the anomaly profile is generated based on the one ormore features.
 30. The network computer of claim 24, wherein theinference engine performs further actions, including, generating theinvestigation profile based on the one or more other portions of thenetwork traffic that are associated with the one or more investigationactivities.